Wiki site synced across six pages (commit 4c8dc09 on the wiki repo) and
issue #2 answered with the shipping conditions for the downstream
workaround removal.
Leaves the wiki-site sync, the issue #2 reply and the version bump
unticked -- those are external deliverables this repository cannot
self-certify, and the pre-merge review was right to flag their absence.
The migration doc is a standing constraint on library design, so an
outdated enum there states an outdated fact. B13's substance survives
the change -- GovDoc's zero was never the problem, the missing label
was -- but the row now names unavailable and the cache_hit-qualified
gap query alongside it.
The reviewer confirmed the T1-T4 ordering holds -- it re-derived every
intermediate state and checked that no construction path can produce
est_tokens=0 with tpm>0 before T4 -- but found four gaps.
Two existing tests go red and the plan never said so: test_types.py:94
asserts the very constraint T4 deletes, and test_embedding.py:105 is a
transport-level case for the fallback T3 rewrites, easy to miss while
looking only at test_openai_compat.py.
The T4 acceptance line claimed all three settlement sides use the
derived value, but the cancel branch never assigns actual and leaves it
at the retry.py:329 initial zero -- an implementer would have "fixed"
a branch the design freezes. Corrected here and in the design section
5 sentence it came from.
USAGE_SOURCES would have landed with no consumer, so T1 now carries the
two value-domain assertions the design asks for, including the one that
pins the no-runtime-validation ruling.
The ordering is the load-bearing part. All three changes interlock and
every wrong interleaving fails silently: flipping the usage fallback to
(0, 0) before the settlement points read the derived value refunds the
whole pre-deduction on success, and flipping the embedding transport
before _merge goes three-state mislabels unavailable batches as
measured. So the plan adds the capability first, moves all five call
sites onto it while it is still equivalent, only then lets the third
state take effect, and unbinds the constraint last.
Registers both wiki entries and links the plan to its design.
Records the approved option, the four rejected alternatives with their
reasons, the intentionally dropped CHS migration item, and the two
defects the independent review caught. Links the entry to m1-core-design
as a refinement, since that milestone is where est_tokens froze with
both jobs attached.
The value-domain table still listed the OCR endpoint as a producer of
"unavailable" while section 3.3 had just decided to keep its "measured"
label -- an implementer following the normative table would have redone
the change that was explicitly dropped, and the guard test would fail.
Also corrects the derivation call-site count to five, qualifies the
retained conservative settlement to the non-dead transient branch only,
and pins the gap metric to "AND cache_hit = false" so cache hits, which
carry cost 0.0 by design, do not inflate it.
The reviewer found two real defects. First, changing the usage fallback
to (0, 0) breaks the success-side settlement too, not just the failure
side: retry.py:338 and embedding.py:271 take actual from the same
return value, so a call whose gateway never sends a usage frame would
have its whole pre-deduction refunded -- systematic TPM undercounting.
Added as change item 9. Second, dropping the OCR item: types.py:51 and
ocr.py:9 both state OCR's zero token count is a fact, not an unknown,
so "measured" was already accurate, and relabelling it would pollute
the very metric used to justify the chosen option.
Also pins the cost short-circuit after the cache_hit branch, confines
value-domain enforcement to producers so no bare ValueError escapes
chat(), completes the authoritative-document list, and narrows the
p90 rejection to the read-port argument.
Split the two jobs SourceConfig.est_tokens has been doing: TPM entry
pre-deduction, where conservative means safe, and the telemetry usage
fallback, where feeding a worst-case upper bound through the output
price inflates cost by ~26x.
Records the approved decisions: usage_source gains an "unavailable"
state whose cost is NULL, and an unset est_tokens derives from
tpm//60 so the in-flight ceiling stays scale-invariant. Also declares
the CHS "conservative accounting" migration item as intentionally
dropped, and the pre-existing global-TPM gap as knowingly unfixed.
Refs: gitea issue #2
Mutation testing showed the negative structured-retries and expected-dim checks
in the env parsing path could be deleted with every test still passing. Their
value is the env key name in the message, so they need tests that assert it.
Changelog now states the real scope of this release and warns that normalising
scope moves the Redis keys, the one change here that silently relocates runtime
state. Records the breaker threshold derivation as deliberately env-only so it
does not resurface as another round.
The verifier found four more env-only behaviours of the same class the branch
was already fixing. The worst is scope: it goes straight into the Redis keys
(pgw:limit:{scope}, pgw:gate:{scope}), so one process using from_env("LLM")
and another constructing scope="LLM" by hand split the rate limit and breaker
state across two namespaces, each tracking its own quota, with no error.
Blank redis_url and pricing_path now collapse to None as from_env has always
done, so they fall into the required-field checks instead of reaching the redis
client as an unparseable URL. EmbeddingSettings gains the __post_init__ it never
had, moving its batch_size and expected_dim checks off the from_env-only path.
Also adds the cache backend whitelist test that mutation testing showed missing.
Round two of the from_env-only validation problem. Fifteen checks still lived
in the env parsing functions: six enum domains, the redis_url requirement for
redis-backed limiter/breaker/cache, cache namespace and TTL, telemetry path and
DSN, non-negative structured retries and non-blank scope. from_settings and
direct construction bypassed all of them.
The five asserts in client.py that claimed config had already validated
redis_url and the telemetry targets now hold on every path, so they revert to
what CLAUDE.md permits: internal invariant declarations that also narrow the
Optional for type checkers. Their comments now name the method that guarantees
them, since the previous wording is exactly what went stale.
Postgres DSNs built by hand now get the SQLAlchemy +driver suffix stripped the
way from_env has always stripped it, with a warning so the rewrite is not
silent. The env path strips earlier, so it stays quiet.
The independent verifier found 15 more checks still living only in from_env:
six enum domains, seven conditional-required pairs and two scalar ranges.
More severe than round one because client.py has five asserts that claim
config already validated the redis_url and telemetry paths, which is false on
the from_settings path.
Also records a normalisation gap the verifier missed: _load_pg_dsn strips the
SQLAlchemy +asyncpg suffix, so a hand-built DSN reaches asyncpg unstripped.
Corrects the changelog claim that the old guard messages only named env keys:
they named fields too, it was the remediation advice that pointed at env keys.
Records the human approval of the design, the mutation-testing evidence from
the independent verifier, and the same-class gap it found: 14 checks (redis_url
presence, telemetry paths, enum validity) still live only in from_env, while
client.py asserts they were already validated. Deliberately out of scope here.
Implementation confirmed that no invalid GatewaySettings instance can exist, so
the factory test's exception fires while evaluating the argument rather than
inside from_settings. Recorded so the test is not misread as the factory
carrying its own validation.
Guards for the three cross-field invariants (source timeout vs lease TTL,
stall window vs max TTFT, probe TTL vs slowest timeout) only ran inside
from_env, so the from_settings path could build a GatewaySettings that
violates the class's own documented invariants. Design moves all of them
plus a non-empty sources check into __post_init__ as _validate_* methods,
matching the existing frozen dataclasses in types.py.
Covers two defects left by PR#1: probe_ttl_s was never moved, and an empty
sources tuple leaked a bare 'max() arg is an empty sequence'.
VT migration abandoned (v1.0 scope becomes two projects), Q1 resolved
to Gitea PyPI, Q6 resolved as judge exemption (unwired zero-consumer
eval scaffolding), migration docs annotated with implementation errata,
and the reference/ read-only rule clarified for the worktree workflow.
AIMD ceiling now respects per-source max_concurrency and the pacer is
assembled explicitly in the client; MIN_CALLS parses as strict int;
acceptance doc corrects source-5 attempt count to 549; design and
migration notes aligned with implemented 429/stall/suppression
semantics and AIMD constants documented.
Round 8 forensics caught the healthy source circuit-opened by five
random empty completions (~20% ambient failure rate makes a 5-streak
land every ~3000 attempts), blacking out the only good source for 60s.
When the window holds min_calls samples below the failure-rate
threshold, a streak is noise and no longer opens the gate; cold-start
and low-traffic semantics are unchanged and sudden death of a warm
source is still caught by the rate channel.
Round 6 hit account-level rate throttling the concurrency AIMD cannot
absorb: at 26 req/min the gateway still returned 16% 429s and each one
burned a third of the retry budget. Retry-After-guided 429s now back
off without consuming attempts (gRPC pushback semantics); the retry
loop gains a per-call ceiling using the same dual-condition stall
verdict as quota-wait (local window exceeded AND no global progress).
Round 5 left three residual failure classes; ladder exhaustion (3.2%
of structured calls with a single re-ask) and 429 leakage (5.8%, AIMD
oscillating above the sustainable point) are addressable: re-ask
default goes 1 to 2 (conservative vs instructor's 3) and the AIMD cut
factor drops to 0.5.
Round 3 showed unconditional yield-after-two-failures pushes the third
attempt onto known-bad sources in heterogeneous pools (83% vs 10%
expected success). OutcomeAwareSelector now exposes health(); a failed
source only yields when some untried candidate scores at least half its
health. Health-blind selectors keep the unconditional rule.
P6 round 2 showed routing convergence turns account-level 429s into
the binding constraint (62% throttle rate at full concurrency). Each
source now carries a local AIMD limit: multiplicative cut on 429,
additive growth on success. Over-limit picks queue via the existing
quota-wait poll instead of burning retry budget or tripping the
circuit-open verdict.
iomgaa