OcrClient is the third telemetry path that skips the chat onion: _emit
builds its own ChatRequest purely to reuse the shared TelemetryEmitter,
so wiring chat() and embed() alone left every OCR row without a tenant
while those rows land in the same llm_calls table. Take the dimensions
at both public entries, validate them there (anything failing further
down is degraded to a warning), and thread them through _call ->
_attempt -> _emit so success, rejection, cancellation and retryable
failure rows all carry the same pair.
EmbeddingClient does not go through the chat onion: it builds its own
ChatRequest inside _emit purely to reuse the shared TelemetryEmitter, so
wiring chat() alone left every embed row without a tenant. Validate the
dimensions at the embed() entry (before batching, since anything failing
further down is degraded to a warning) and thread them through
_embed_batch -> _attempt -> _emit so every batch row carries the same
pair.
Validation runs before the request enters the onion: every failure inside it
is downgraded to a warning by the telemetry layer, so validating in there
would not validate anything.
The dimensions stay out of the cache key — cache_namespace already carries
tenant isolation, and folding meta in would cold-start every existing entry.
Both telemetry backends gain tenant_id and meta at the end of the
column list, and TelemetryEmitter fills them from the request. The two
halves ship together because the emitter is the only caller of
record_llm_call: adding the columns without filling them leaves every
row short of two keys, and the backends read those keys outside their
try block, so the KeyError degrades to a warning and the whole table
stops filling.
The columns are appended, never inserted. An old table can only gain
columns through ALTER, which puts them last; a new table built from the
DDL would put them wherever the DDL says. Anywhere but the end and the
two paths produce different physical column orders, while the INSERT
uses positional placeholders.
The two backends spell the default differently for different reasons.
SQLite refuses a NOT NULL column without a non-NULL constant default
outright, so the default is what makes the backfill legal at all. On
Postgres a non-volatile constant default is what keeps the ALTER from
rewriting the table, and NOT NULL DEFAULT '' is what keeps old rows out
of the black hole a NULL tenant_id falls into under an RLS policy.
Normalisation happens in the emitter, not the recorder, matching how
canonical_sampling_json already settles the sampling column: None
becomes the empty string, an empty mapping becomes the literal '{}'.
Keys are sorted so one set of dimensions serialises identically on
every row, and allow_nan=False is a second gate behind the entry
validation -- json.dumps would otherwise write a bare NaN, which JSONB
rejects, and the failed insert would be swallowed as a warning.
All three emit entry points read the request. Cache hits read it too,
rather than the replayed response: the dimensions answer who made this
call, not who made the one whose result is being replayed.
Adds validate_caller_dimensions and the two ChatRequest fields that
carry them. Every limit rejects rather than trims: Langfuse drops
metadata values past 200 characters, which leaves the caller believing
something was recorded when nothing was.
Whitespace on tenant_id is refused outright instead of stripped. " t1"
and "t1" compare unequal inside an RLS policy, so silently rewriting the
caller's value would hand them a tenant whose rows they cannot find.
Non-finite floats are refused for a concrete reason: json.dumps writes
them as the bare literals NaN and Infinity, which are not valid JSON and
which JSONB rejects. Letting one through turns a caller's input mistake
into a failed insert, and the telemetry layer degrades failed inserts to
a warning -- so the mistake would surface as missing rows, nowhere else.
The new fields go after sampling so no positional construction of
ChatRequest shifts. Validation is split across three helpers to keep
each one under the complexity gate.
Verifier mutation test: flipping _translate_429 to parse the summary
left all 824 tests green. The padding was one long string value, so the
cut landed inside it - and head-and-tail retention kept the trailing
error object, leaving the summary parseable. Many keys put the cut
between structural tokens, where the summary stops being valid JSON.
Mutation now fails as it should. Also splits OCR 429 out on its own.
Issue #10 Task 5, the acceptance claim. Before the fix this asserted
against 'qwen_1 请求被拒: 400' and failed on the first substring - which
is exactly what the downstream batch was left with. Uses the real body
from the issue, and checks the trailing code too, since a head-only cut
would drop the one field you quote when chasing the provider.
Issue #10 Task 4: the OCR side said only 'HTTP 404'. The issue reported
the chat path, but the batch that lost its 400 was reading tables - the
same blind spot, one transport over. Reuses the shared summarizer.
Issue #10 Task 3: five branches each built their own message, so adding
the summary would have meant five copies. Table-driven classification
composes it in one place instead, and the 429 split still parses the
untruncated body - reading the summary would demote an oversized
insufficient_quota to a plain rate limit and stop force_open.
Issue #10 Task 2: head-and-tail rather than a head-only cut, because the
code and request_id that let you chase the provider sit at the very end
of a JSON error body. Cap 2048 follows k8s client-go for the same job.
Issue #10 Task 1: a rejected call's reason had nowhere to live. The
field goes on the base class because these errors all come from one HTTP
response - which class it is and what the peer said are orthogonal.
PostgreSQL checks the schema CREATE privilege before the IF NOT EXISTS
existence test, so an account with only table-level INSERT was denied on
CREATE TABLE IF NOT EXISTS even though the table was right there and
writable. The denial set _failed and the whole recorder went no-op for
the process lifetime, silently: 150+ calls downstream lost their latency,
token and cost rows with nothing but one warning to show for it.
The probe is the direct fix. The larger fix is the criterion: structural
degradation now means "provably cannot write" (pool creation failed, or
the table is absent and cannot be created), not "something threw during
init" -- a probe or acquire failure just skips the row and retries on the
next call.
SQLite stays as it is on purpose. Measured: it short-circuits the
statement at parse time, so it passes even under another connection's
EXCLUSIVE lock or on a read-only file. A probe there would buy nothing;
the docstring now says so to keep symmetry-minded future edits away.
Patch rather than minor: the error surface is unchanged and no public
signature moved. What downstream must notice is timing, not types — the
worst-case call duration rises to roughly max_attempts * timeout_s now
that the retry budget actually applies.
Pre-release review caught an overreaching promise in the changelog entry:
the 429 bound holds only when the stall verdict can fire at all, i.e. when
the whole scope has no progress. The verdict is a conjunction, so a call
does not die while other calls in the scope are still producing — by
design — which leaves no hard per-call ceiling in that case. That property
predates this fix and is now stated with its precondition instead of as an
unconditional guarantee.
The wiki sync in the release checklist is a no-op again: the doc site has
been down since 2026-08-02 and its landing page names CHANGELOG.md as the
version source of truth, which this commit updates.
Independent verification found the first cut had swapped one bug for a
worse one. The budgets were split by "did we send a request", so a 429
attempt counted as productive — but 429 is exempt from the retry budget,
so its time burned neither budget. Against a queueing gateway that holds
the request for the full timeout before answering 429, a call could hang
for 301 attempts / 25.2 hours, measured, versus 301 seconds before the
change.
The split is now by which budget the time consumes: time that burns
max_attempts is excluded from stall, time that does not (429 attempts
included) belongs to stall. Measured again: back to one attempt / 301s.
Only the chat loop needs this — embedding and ocr count 429 against
max_attempts unconditionally, so the gap never existed there. The stall
verdict moved into _stalled(), which both call sites had duplicated, to
keep __call__ under the complexity gate.
The concurrency case used two RetryMW instances, so instance-level sharing
was hidden by object isolation and a clock promoted to an instance
attribute passed all seven cases. Both cases now reuse one mw, and a new
one idles past the window between two calls on that instance — the shape
that would expose _entered_at pinned to process start. Mutation-checked:
promoting the clock fails the new case.
Same failure path as the embedding loop: one timed-out attempt drains the
wall-clock window, and the next round without a runnable source declares
the scope dead in _on_no_runnable. All three governance loops now meter
stall the same way.
The embedding loop shares the wall-clock entered_at and the same stall
verdict, so it failed the same way through a different path: one timed-out
attempt, then any round with no runnable source, and _on_no_runnable
declared the scope dead. Issue #8 only recorded the chat path; the
regression test pins this one.
Issue #8: with timeout_s >= stall_window_s a single timed-out request
exhausted the stall window before the second attempt was even dispatched,
so LLM_MAX_RETRIES never applied and the whole scope was declared dead.
Root cause is that real attempts and non-productive waiting charged the
same wall clock, while the stall budget is the smaller of the two. The new
StallClock subtracts attempt time from the stall account, leaving the two
budgets orthogonal: attempts bill max_attempts, waiting bills
stall_window_s. The dual-condition verdict, the inf semantics of
progress_age_s, the 429 exemption and the error surface are untouched.
The productive boundary is _attempt itself, telemetry included, so a slow
recorder cannot push a call into a stalled verdict.
Letting SourceNotConfiguredError through the gate wrappers opened a hole
the recheck caught: _record_quietly only degrades GovernanceBackendError,
so an assembly defect raised from the accounting side would now escape and
destroy a response from a call that had already genuinely succeeded. That
inverts the exact invariant _record_quietly exists to hold.
Widening _record_quietly is the right fix rather than narrowing the
wrappers, because that layer degrades by what the path is (accounting, the
call is already done) rather than by which error type shows up. Narrowing
would have left 4 of 9 wrapper methods as exceptions to a rule nobody can
remember.
No backend raises it from an accounting method today, so this is a
guardrail for whoever adds source-name validation to a breaker backend.
The stub that first reported this green was wrong: its record_success
lacked count_attempt, so it raised TypeError and the wrapper relabeled it.
Fixed signature, then the test failed as it should have.
Also finishes the three-to-five leak path correction across the four
remaining spots, including the wiki summary card that indexes this design.
Independent verification caught that the split shipped in the previous
commit did not actually hold on the only path production uses. The gate
wrappers re-raise GovernanceBackendError but nothing else, so
SourceNotConfiguredError fell into the following `except Exception` and
came back out as a governance_backend_down failure with retry_after_s=5.0.
A misconfigured source name would still retry forever and never surface.
The existing tests missed it because both of them call the private _cfg()
directly, one layer below the wrapper the governance loops actually go
through. The regression test goes through QuotaGate.
telemetry.py has to widen its terminal catch in the same commit: once the
wrapper stops relabeling the error, it is no longer a GovernanceBackendError,
and it is raised before any attempt exists, so the path would have recorded
no telemetry at all.
Also corrects the leak path count from three to five. QuotaGate.stats and
BreakerGate.retry_after_s are not wrapped by _record_quietly either.
A fail-closed limiter or breaker backend means the scope cannot emit a
single request, which is exactly scope-level unavailability. But the error
sat directly under PolyGatewayError, so a caller writing only
`except GatewayUnavailableError` dropped it into the catch-all branch:
Redis blips once and a backlog of tasks burns its business failure budget
into the dead letter queue, over a fault a restart would clear.
Three gate paths leak to callers rather than being absorbed by
_record_quietly (try_acquire, try_enter, progress_age_s); each is now
pinned by a test, since none of them had one before.
The two unknown-source sites move to SourceNotConfiguredError instead of
following along. They report a misconfigured source name, not an outage,
and letting them into the retryable family would be the mirror of the bug
being fixed here: the task would retry forever and never surface.
Pure addition ahead of the reparenting, so this commit leaves every
existing caller and test untouched.
SourceNotConfiguredError deliberately stays outside GatewayUnavailableError:
a source name that is not in the limiter's config dict is an assembly
defect, not a transient outage, and folding it into the retryable family
would let a typo retry forever without ever reaching a dead letter queue.
The retry_after_s default is 5.0 rather than 0 because a backlog released
at zero delay would stampede a backend that is already down.
The thinking matrix had been running inside make ci all along, which is
not what the design claimed. It takes seven minutes, spends 137 real
calls, and its criteria are statistical, so a network hiccup fails the
build for reasons unrelated to the change under test -- one run died on
three consecutive network errors exhausting the source.
The project already has the mechanism for this: the slow marker, which
addopts excludes by default and the config comments describe as "CI runs
it on demand". Marking the matrix slow brings make ci back down from
seven minutes to ninety seconds while the matrix stays a merge
requirement via -m slow.
The design also claimed e2e does not run in CI. It does: make test runs
pytest over tests/, e2e included, and the existing smoke tests really
call the gateway whenever .env has credentials. Only slow-marked tests
are excluded. Both documents now say so.
Version sources are pyproject and __init__; a test enforces they agree,
and it caught the second one being missed.
The verifier caught that the disable-direction evidence only proved "no
regression", not "actually took effect": on M3 the disabled runs and the
no-opinion baseline are identically distributed, because that model does
not reason by default anyway. So the disable runs alone cannot rule out
the very failure mode issue #5 is about -- the parameter being silently
dropped upstream. The bogus-value experiment that does rule it out was
sitting in the findings document instead of the test suite; it is now
case L3b, and the L3 assertion that could never fail is gone.
Also from the review: the e2e helper caught bare Exception, which would
have disguised a library bug as an unavailable source, exactly the
silence the reporting discipline exists to prevent; the unregistered
model warning fired on every request instead of once per source; and the
transport caught ValueError broadly enough to mislabel unrelated errors,
now narrowed to a dedicated ThinkingUnsupportedError.
The design and plan still described the original judgement criteria,
which the measurements had already overturned. Both now match what the
tests actually do, and the design no longer claims the only new failure
surface is the openai one -- dissect configures MiniMax-M2.7 with
ENABLE_THINKING=false and will fail at assembly, which has to be
coordinated before this merges.
A sixteen-row matrix over 127 real calls: disable and enable on
MiniMax-M3 in both streaming and non-streaming mode, extra_body winning
over the profile slot, qwen and deepseek still disabling correctly, a
drift sentinel that re-derives every registered capability from live
behaviour, and the assembly guard refusing the models that cannot
comply.
Two judgement criteria had to be corrected by the data they were meant
to judge. Output length cannot separate the two regimes at all -- the
disabled runs reach 46 tokens when the model narrates its working in
the visible answer, and the enabled runs drop to 13 when medium effort
barely thinks. reasoning_tokens separates them cleanly in both
directions, which is precisely what issue #6 was collected for. A
second anchor compares prompt_tokens between the two regimes: the
vendor injects a reasoning instruction when thinking is on, so the
input side grows, and comparing the two runs relatively avoids
hardcoding any vendor number.
Provider names are mapped explicitly rather than guessed from the model
string; guessing had silently skipped the qwen row behind a "source
unavailable" reason that was not true.
enable_thinking=False was a no-op for minimax and openai sources: both
profiles had empty dicts on each side, so the payload update injected
nothing while the caller believed reasoning had been turned off. A
downstream project was blocked on exactly this.
The root cause is that an empty dict meant two different things -- "no
injection needed" and "we do not know how this provider spells it" --
and that a provider-level table cannot express what turned out to be a
per-model property. Live testing showed MiniMax-M3 can disable
reasoning via reasoning_effort while M2.7 and M2.5 cannot be disabled
at all, which two external registries independently confirm.
So the shape stays at provider level and a capability table joins it at
model level. Unknown, unsupported and no-opinion are now three distinct
values, and resolve_thinking is the single place they meet: it raises at
assembly time when a model cannot honour the request, warns and injects
for unregistered models, and injects silently otherwise. Every registered
capability carries the evidence it was derived from.
enable_thinking also joins the cache fingerprint, since it now really
does change the request body.
Reasoning tokens are already counted inside completion_tokens, so the
cost total was never wrong -- what was missing is the attribution: how
much of a call was spent thinking rather than answering.
LLMResponse and TransportResult each gain a trailing reasoning_tokens
field, and the telemetry port grows from 21 to 22 columns with the new
column appended in both backends so fresh and migrated schemas keep the
same physical order.
None means this particular call did not report the field, not that the
source never reports it: a relay that falls back to a local tokenizer
replaces the whole usage object and drops completion_tokens_details.
Downstream checks must therefore read "in (None, 0)"; no provider was
observed reporting a literal zero.
The test passed overlay and sampling as separate objects while production
aliases them, so an in-place mutation slipped through it. Also syncs the
telemetry schema page and adds the missing postgres round-trip assertion.
The gate check forced operators to guess a per-call token size before
they could enable the TPM gate at all; est_tokens is now an optional
tuning override and effective_est_tokens() derives the reservation from
the provider quota. Reservation and settlement already read the same
derived value, so the deposit still nets to zero on both the success
path and the non-dead transient failure path.
The rest of _validate_gates is untouched, and the est_tokens field plus
its EST_TOKENS env key stay put for migration compatibility.
Task 1 of the est_tokens decoupling: capability only, no call site
touched, so library behaviour is unchanged word for word.
SourceConfig.effective_est_tokens() returns the explicit est_tokens when
set, otherwise tpm // 60 floored at 1, otherwise 0 when the TPM gate is
off. The divisor is scale free: any quota size yields the same in-flight
ceiling of roughly sixty calls, which is what makes the default
explainable where a fixed constant was not.
USAGE_SOURCES lands with the two assertions the design asks for, not as
a dead constant. test_usage_source_domain.py drives every production
point -- _resolve_usage, _resolve_embedding_usage, _merge and the three
TelemetryEmitter.emit_* helpers -- and asserts the output stays inside
the domain; it is a separate file because the assertion spans
transports, embedding and telemetry, and the innermost kernel test
should not depend on implementations. The second assertion pins the
opposite ruling: constructing LLMResponse with an out-of-domain value
must not raise, since a bare ValueError at a runtime construction point
falls outside the four error categories and would escape chat().
tpm > 0 with est_tokens = 0 is still rejected until Task 4, so the
derivation tests build the future-legal shape through a helper that
bypasses the constraint; the helper collapses back to _make_source once
the constraint is gone.
iomgaa