The transport now hands back the tier it actually sent, and that tier
rides TransportResult into LLMResponse. It is not the requested one:
under EFFORT_FALLBACK=nearest a medium request goes out as low, and
telemetry grouping by the requested tier would file the row under a tier
that never left the process.
Reconciliation judges the same tier instead of the old enable_thinking
bool, and the warning throttle keys on it. Keyed on the bool, every tier
of one model shared a single key, so the second contradiction was
silenced for the lifetime of the transport. The predicate is an identity
check against Effort.NONE on purpose -- the member's value is the
non-empty string "none", so any truthiness test would send every strength
tier down the "asked to disable" branch and invert the alarm.
Both public assembly paths took the tier on trust: a bare "none" from
JSON or a hand-built SourceConfig stayed a str, and `is Effort.NONE`
then read it as a contradiction and crashed on `.value` while wording
the error -- the caller got an AttributeError where a ValueError was
promised, and on the request side that unclassified exception walked
straight through the transport's ThinkingUnsupportedError catch and the
retry classifier.
Normalize at the two entrances instead, matching what the .env path has
always done, and let EFFORT_FALLBACK be spelled with the same freedom as
its neighbour.
The three-layer priority (call > source > enable_thinking sugar > silence)
now lives in one pure function, thinking.effective_effort(). The assembly
guard and the request hot path used to each carry their own inline copy of
the sugar conversion; two copies of the same judgement drift into the worst
shape there is - passes at assembly, raises at runtime.
The guard now also honours effort_fallback, so a source that opted into
nearest is no longer sentenced at assembly for a tier it could have mapped.
The boolean could say a model reasons or does not. It could not say what
GLM-5.3 and Gemini 3 Pro actually do: refuse to stop reasoning while
still letting you ask for less. So capability becomes the list of tiers a
model serves, and `none`'s presence in it is what "can_disable" now reads.
Effort carries `auto` alongside the strength tiers. Nine of the models on
our gateway are pure switches with no tier to name, and without `auto`
they would have to borrow a strength tier to mean "on" — which is the
exact bug this work exists to remove.
Tiers land as documented guesses from four registries that agree; every
entry says so in its evidence, and task 10 replaces them with measurements.
The reasoning_tokens docstring was still teaching downstream to treat
None or 0 as no reasoning. The changelog and the schema page had both
been corrected; the docstring had not, and it is the copy that ships in
the wheel and shows up on hover. Someone writing a report from it would
have counted every real MiniMax reasoning call as not reasoning, which
is issue #16 all over again with the tests green.
The original wording stays, since reading pre-1.3.1 rows still needs
it. What follows it now says when it expired and what to read instead.
Two more places had drifted the same way: the changelog and the
architecture doc described the throttle and the cache fallback as they
were before this review, which is to say as the opposite of what the
code now does.
The claim that the two throttle sets would suppress each other does not
survive checking, as the mutation testing showed: their key spaces do
not overlap. Keeping them apart is still right, but for the honest
reason, which is that the two warnings have unrelated lifetimes.
Both assembly paths fill it, streaming and non-streaming alike. Filling
only one is exactly the divergence this issue exposed: M3 returns
reasoning prose over SSE and nothing at all over the plain endpoint, so
a verdict computed on one path says nothing about the other.
The field defaults to UNKNOWN on both TransportResult and LLMResponse.
A transport that does not judge should not get to declare absence on
the provider's behalf, and a default that stays silent is the only one
that cannot lie.
reasoning_tokens=None has been carrying two meanings at once, no
reasoning and no report, and the library resolved the ambiguity by
quietly claiming the first. ThinkingObservation splits them: UNKNOWN
says the call left no signal, ABSENT says the provider reported zero.
The verdict ranks evidence by hardness. Reasoning prose is the fact
itself; reasoning_tokens is a report about the fact, so a missing
report cannot overrule prose that is right there. The prose check
strips first, since a gateway that returns whitespace is not evidence.
The enum lives in types.py, not in the new thinking.py, because
LLMResponse is typed on it and the innermost layer must not import a
decision module.
Telemetry degradation used to be a single warning and a private boolean.
In a long-running process that is indistinguishable from telemetry working:
issue #15 was only found by hand-reconciling milestone log lines against
llm_calls rows, after 19 calls had silently gone unrecorded. The SQLite
side was worse — once init failed, every write returned without even a
log line.
Degradation now has one shared owner. TelemetryStatusTracker holds the
state machine (enter/recover/drop/should-retry), announces entry and
recovery once each, and repeats the drop count under a row-and-time
double threshold so a degraded backend neither floods the log nor goes
quiet. Both recorders hold one; both count the rows they drop.
For programmatic consumers, TelemetryStatus is a frozen snapshot exposed
as telemetry_status on all three clients, resolved through a single
isinstance check. It is a separate optional port rather than a member of
TelemetryRecorder: that protocol is @runtime_checkable, so adding an
attribute would make every implementation that only defines
record_llm_call stop satisfying it — downstream isinstance assertions
would break on upgrade. The existing assertion in test_ports.py is what
keeps that decision honest.
Failure criteria are deliberately untouched here: Postgres still treats a
pool failure as permanent, only now visibly. `_failed` and the tracker
therefore both carry the verdict for the span of this one change; the
cooldown rework collapses them into the tracker alone.
The key-count cap exists to catch a whole request body dumped into meta.
That is exactly the shape where the per-key regex runs tens of thousands
of times before the real reason surfaces, so the cheap check goes first.
Also correct two stale docstrings: postgres.py still claimed 22 columns
(it is 24), and _canonical_meta_json promised to raise on non-finite
floats. It is evaluated inside _record's degradation try, so the real
outcome is a warning plus a dropped row -- never an error the caller
sees. What the gate actually buys us is the SQLite side, whose meta is a
TEXT column that would happily store a literal NaN.
Adds validate_caller_dimensions and the two ChatRequest fields that
carry them. Every limit rejects rather than trims: Langfuse drops
metadata values past 200 characters, which leaves the caller believing
something was recorded when nothing was.
Whitespace on tenant_id is refused outright instead of stripped. " t1"
and "t1" compare unequal inside an RLS policy, so silently rewriting the
caller's value would hand them a tenant whose rows they cannot find.
Non-finite floats are refused for a concrete reason: json.dumps writes
them as the bare literals NaN and Infinity, which are not valid JSON and
which JSONB rejects. Letting one through turns a caller's input mistake
into a failed insert, and the telemetry layer degrades failed inserts to
a warning -- so the mistake would surface as missing rows, nowhere else.
The new fields go after sampling so no positional construction of
ChatRequest shifts. Validation is split across three helpers to keep
each one under the complexity gate.
Reasoning tokens are already counted inside completion_tokens, so the
cost total was never wrong -- what was missing is the attribution: how
much of a call was spent thinking rather than answering.
LLMResponse and TransportResult each gain a trailing reasoning_tokens
field, and the telemetry port grows from 21 to 22 columns with the new
column appended in both backends so fresh and migrated schemas keep the
same physical order.
None means this particular call did not report the field, not that the
source never reports it: a relay that falls back to a local tokenizer
replaces the whole usage object and drops completion_tokens_details.
Downstream checks must therefore read "in (None, 0)"; no provider was
observed reporting a literal zero.
The gate check forced operators to guess a per-call token size before
they could enable the TPM gate at all; est_tokens is now an optional
tuning override and effective_est_tokens() derives the reservation from
the provider quota. Reservation and settlement already read the same
derived value, so the deposit still nets to zero on both the success
path and the non-dead transient failure path.
The rest of _validate_gates is untouched, and the est_tokens field plus
its EST_TOKENS env key stay put for migration compatibility.
Task 1 of the est_tokens decoupling: capability only, no call site
touched, so library behaviour is unchanged word for word.
SourceConfig.effective_est_tokens() returns the explicit est_tokens when
set, otherwise tpm // 60 floored at 1, otherwise 0 when the TPM gate is
off. The divisor is scale free: any quota size yields the same in-flight
ceiling of roughly sixty calls, which is what makes the default
explainable where a fixed constant was not.
USAGE_SOURCES lands with the two assertions the design asks for, not as
a dead constant. test_usage_source_domain.py drives every production
point -- _resolve_usage, _resolve_embedding_usage, _merge and the three
TelemetryEmitter.emit_* helpers -- and asserts the output stays inside
the domain; it is a separate file because the assertion spans
transports, embedding and telemetry, and the innermost kernel test
should not depend on implementations. The second assertion pins the
opposite ruling: constructing LLMResponse with an out-of-domain value
must not raise, since a bare ValueError at a runtime construction point
falls outside the four error categories and would escape chat().
tpm > 0 with est_tokens = 0 is still rejected until Task 4, so the
derivation tests build the future-legal shape through a helper that
bypasses the constraint; the helper collapses back to _make_source once
the constraint is gone.
Threshold auto-raise now uses per-source concurrency only (the M2
global-concurrency formula neutered the breaker at scale). Four new
optional keys: MIN_CALLS, FAIL_RATE, WINDOW_S, MAX_COOLDOWN_S.
Includes config aggregation for multi-source env keys, from_env and
from_settings factories with explicit shared-backend injection,
gather_bounded, top-level exports, tightened import-linter layers with
the gate removed from the Makefile, and the finalized .env.example.
iomgaa