Task 1 of the est_tokens decoupling: capability only, no call site
touched, so library behaviour is unchanged word for word.
SourceConfig.effective_est_tokens() returns the explicit est_tokens when
set, otherwise tpm // 60 floored at 1, otherwise 0 when the TPM gate is
off. The divisor is scale free: any quota size yields the same in-flight
ceiling of roughly sixty calls, which is what makes the default
explainable where a fixed constant was not.
USAGE_SOURCES lands with the two assertions the design asks for, not as
a dead constant. test_usage_source_domain.py drives every production
point -- _resolve_usage, _resolve_embedding_usage, _merge and the three
TelemetryEmitter.emit_* helpers -- and asserts the output stays inside
the domain; it is a separate file because the assertion spans
transports, embedding and telemetry, and the innermost kernel test
should not depend on implementations. The second assertion pins the
opposite ruling: constructing LLMResponse with an out-of-domain value
must not raise, since a bare ValueError at a runtime construction point
falls outside the four error categories and would escape chat().
tpm > 0 with est_tokens = 0 is still rejected until Task 4, so the
derivation tests build the future-legal shape through a helper that
bypasses the constraint; the helper collapses back to _make_source once
the constraint is gone.
The verifier found four more env-only behaviours of the same class the branch
was already fixing. The worst is scope: it goes straight into the Redis keys
(pgw:limit:{scope}, pgw:gate:{scope}), so one process using from_env("LLM")
and another constructing scope="LLM" by hand split the rate limit and breaker
state across two namespaces, each tracking its own quota, with no error.
Blank redis_url and pricing_path now collapse to None as from_env has always
done, so they fall into the required-field checks instead of reaching the redis
client as an unparseable URL. EmbeddingSettings gains the __post_init__ it never
had, moving its batch_size and expected_dim checks off the from_env-only path.
Also adds the cache backend whitelist test that mutation testing showed missing.
The warning added earlier in this branch logged the whole Postgres DSN, password
included, and nothing else in the library has ever printed a connection string.
It now reports only the scheme segment, which is the part that actually changed.
Regression test asserts the password and host/path never reach the log.
Round two of the from_env-only validation problem. Fifteen checks still lived
in the env parsing functions: six enum domains, the redis_url requirement for
redis-backed limiter/breaker/cache, cache namespace and TTL, telemetry path and
DSN, non-negative structured retries and non-blank scope. from_settings and
direct construction bypassed all of them.
The five asserts in client.py that claimed config had already validated
redis_url and the telemetry targets now hold on every path, so they revert to
what CLAUDE.md permits: internal invariant declarations that also narrow the
Optional for type checkers. Their comments now name the method that guarantees
them, since the previous wording is exactly what went stale.
Postgres DSNs built by hand now get the SQLAlchemy +driver suffix stripped the
way from_env has always stripped it, with a warning so the rewrite is not
silent. The env path strips earlier, so it stays quiet.
Patch release for the settings invariant fix. The changelog carries a separate
'behaviour tightening' section because a patch number gives downstreams no
warning that construction can now raise where it previously did not.
The lease, stall and probe-TTL guards only ran inside GatewaySettings.from_env,
so from_settings() and direct construction could produce settings that violate
the class's own invariants: the permit lease could expire mid-request (silently
exceeding the concurrency quota), a normal slow first token could be killed as a
stall, and a half-open probe could be taken over while still in flight.
Guards move into __post_init__ as _validate_* methods, matching every frozen
dataclass in types.py, so all six factories plus dataclasses.replace are covered
by one check. Adds a non-empty sources invariant that previously only from_env
enforced. Messages now name fields instead of env keys, since callers who build
settings by hand never set those keys.
AIMD ceiling now respects per-source max_concurrency and the pacer is
assembled explicitly in the client; MIN_CALLS parses as strict int;
acceptance doc corrects source-5 attempt count to 549; design and
migration notes aligned with implemented 429/stall/suppression
semantics and AIMD constants documented.
Round 8 forensics caught the healthy source circuit-opened by five
random empty completions (~20% ambient failure rate makes a 5-streak
land every ~3000 attempts), blacking out the only good source for 60s.
When the window holds min_calls samples below the failure-rate
threshold, a streak is noise and no longer opens the gate; cold-start
and low-traffic semantics are unchanged and sudden death of a warm
source is still caught by the rate channel.
Round 6 hit account-level rate throttling the concurrency AIMD cannot
absorb: at 26 req/min the gateway still returned 16% 429s and each one
burned a third of the retry budget. Retry-After-guided 429s now back
off without consuming attempts (gRPC pushback semantics); the retry
loop gains a per-call ceiling using the same dual-condition stall
verdict as quota-wait (local window exceeded AND no global progress).
Round 5 left three residual failure classes; ladder exhaustion (3.2%
of structured calls with a single re-ask) and 429 leakage (5.8%, AIMD
oscillating above the sustainable point) are addressable: re-ask
default goes 1 to 2 (conservative vs instructor's 3) and the AIMD cut
factor drops to 0.5.
Round 4 showed tail placement routes the third attempt to junk sources
whenever the credible alternative is gate-skipped (tight-RPM source
admitted as credible, then skipped by the limiter, falling through to
the flapping watchdog source). Demoted sources now sit between credible
and non-credible candidates.
Round 3 showed unconditional yield-after-two-failures pushes the third
attempt onto known-bad sources in heterogeneous pools (83% vs 10%
expected success). OutcomeAwareSelector now exposes health(); a failed
source only yields when some untried candidate scores at least half its
health. Health-blind selectors keep the unconditional rule.
P6 round 2 showed routing convergence turns account-level 429s into
the binding constraint (62% throttle rate at full concurrency). Each
source now carries a local AIMD limit: multiplicative cut on 429,
additive growth on success. Over-limit picks queue via the existing
quota-wait poll instead of burning retry budget or tripping the
circuit-open verdict.
RetryMW keeps a per-call failure map (local, never instance state):
a source failing twice in one call yields to the next candidate.
Attempt outcomes feed OutcomeAwareSelector behind a swallow-and-warn
guard; ResultInvalid and provider-rejected paths record success with
count_attempt=False so the breaker window stays clean. Same accounting
applied in EmbeddingClient.
Score is success-rate EWMA over (1 + inflight) with a 0.05 exploration
floor so quarantined sources can prove recovery; EWMA climb doubles as
slow-start. New optional OutcomeAwareSelector port feeds attempt
outcomes.
Dual-channel opening: consecutive failures (CHS-compatible, no streak
bump) plus windowed failure rate (two 30s buckets, min_calls guard).
429s bypass both channels as backpressure, and a probe hitting 429
releases instead of holding the lease. Open duration doubles per
rate/probe reopen up to max_cooldown_s, decaying after stable CLOSED.
record_success gains count_attempt so bad-result successes stay out of
the window.
Threshold auto-raise now uses per-source concurrency only (the M2
global-concurrency formula neutered the breaker at scale). Four new
optional keys: MIN_CALLS, FAIL_RATE, WINDOW_S, MAX_COOLDOWN_S.
Classify empty completions as transient per human ruling (fixes flaky
real-gateway smoke and prevents caching empty responses), rename the
factory injection parameter gate to breaker per the frozen design,
rewrite the probe-entry cleanup without except BaseException, declare
python-dotenv explicitly, add a mid-backoff cancellation test, and
record all implementation errata in the design and architecture docs.
Includes config aggregation for multi-source env keys, from_env and
from_settings factories with explicit shared-backend injection,
gather_bounded, top-level exports, tightened import-linter layers with
the gate removed from the Makefile, and the finalized .env.example.
iomgaa