Reasoning tokens are already counted inside completion_tokens, so the
cost total was never wrong -- what was missing is the attribution: how
much of a call was spent thinking rather than answering.
LLMResponse and TransportResult each gain a trailing reasoning_tokens
field, and the telemetry port grows from 21 to 22 columns with the new
column appended in both backends so fresh and migrated schemas keep the
same physical order.
None means this particular call did not report the field, not that the
source never reports it: a relay that falls back to a local tokenizer
replaces the whole usage object and drops completion_tokens_details.
Downstream checks must therefore read "in (None, 0)"; no provider was
observed reporting a literal zero.
The gate check forced operators to guess a per-call token size before
they could enable the TPM gate at all; est_tokens is now an optional
tuning override and effective_est_tokens() derives the reservation from
the provider quota. Reservation and settlement already read the same
derived value, so the deposit still nets to zero on both the success
path and the non-dead transient failure path.
The rest of _validate_gates is untouched, and the est_tokens field plus
its EST_TOKENS env key stay put for migration compatibility.
Task 1 of the est_tokens decoupling: capability only, no call site
touched, so library behaviour is unchanged word for word.
SourceConfig.effective_est_tokens() returns the explicit est_tokens when
set, otherwise tpm // 60 floored at 1, otherwise 0 when the TPM gate is
off. The divisor is scale free: any quota size yields the same in-flight
ceiling of roughly sixty calls, which is what makes the default
explainable where a fixed constant was not.
USAGE_SOURCES lands with the two assertions the design asks for, not as
a dead constant. test_usage_source_domain.py drives every production
point -- _resolve_usage, _resolve_embedding_usage, _merge and the three
TelemetryEmitter.emit_* helpers -- and asserts the output stays inside
the domain; it is a separate file because the assertion spans
transports, embedding and telemetry, and the innermost kernel test
should not depend on implementations. The second assertion pins the
opposite ruling: constructing LLMResponse with an out-of-domain value
must not raise, since a bare ValueError at a runtime construction point
falls outside the four error categories and would escape chat().
tpm > 0 with est_tokens = 0 is still rejected until Task 4, so the
derivation tests build the future-legal shape through a helper that
bypasses the constraint; the helper collapses back to _make_source once
the constraint is gone.
Threshold auto-raise now uses per-source concurrency only (the M2
global-concurrency formula neutered the breaker at scale). Four new
optional keys: MIN_CALLS, FAIL_RATE, WINDOW_S, MAX_COOLDOWN_S.
Includes config aggregation for multi-source env keys, from_env and
from_settings factories with explicit shared-backend injection,
gather_bounded, top-level exports, tightened import-linter layers with
the gate removed from the Makefile, and the finalized .env.example.
iomgaa