Bumping the version is not releasing. 1.0.6 and 1.1.0 both got a version
bump and a changelog entry but were never uploaded, so the registry sat at
1.0.5 and downstream could not install any of those fixes.
The ordering matters in one non-obvious way: README has to be correct
before the build, because sdist freezes whatever is there at that moment.
That is exactly how 1.1.1 shipped with a stale README. The install pin is
called out by name since it is the easiest line to forget and the most
damaging to leave wrong.
Also records where the Gitea token actually lives — tea's config, not
.pypirc — after that misreading led to a wrong "no credentials" claim.
Three README drifts, none of them about issue #8 alone:
- the telemetry row said 18 fields; record_llm_call takes 22 (verified by
inspect.signature). ports.py claimed 20 in its own docstring, so both
records of the same fact were stale.
- LLMResponse.cost is hardcoded to None on the chat path (retry.py:519).
Cost only ever reaches telemetry. The old doc site named this as a known
trap, so the capability row now says it outright.
- backpressure had no row at all, which is what issue #8 was about.
The pin still said ==1.0.*, which caps downstream at 1.0.5 and hides
every fix since. Now that 1.1.1 is actually in the registry the pin can
move; it was missed when 1.1.0 was tagged.
Patch rather than minor: the error surface is unchanged and no public
signature moved. What downstream must notice is timing, not types — the
worst-case call duration rises to roughly max_attempts * timeout_s now
that the retry budget actually applies.
Pre-release review caught an overreaching promise in the changelog entry:
the 429 bound holds only when the stall verdict can fire at all, i.e. when
the whole scope has no progress. The verdict is a conjunction, so a call
does not die while other calls in the scope are still producing — by
design — which leaves no hard per-call ceiling in that case. That property
predates this fix and is now stated with its precondition instead of as an
unconditional guarantee.
The wiki sync in the release checklist is a no-op again: the doc site has
been down since 2026-08-02 and its landing page names CHANGELOG.md as the
version source of truth, which this commit updates.
Registration pages carry the chosen approach, why the split is by "which
budget the time consumes", the five rejected alternatives with reasons,
and the 3.6 correction found during independent verification.
Independent verification found the first cut had swapped one bug for a
worse one. The budgets were split by "did we send a request", so a 429
attempt counted as productive — but 429 is exempt from the retry budget,
so its time burned neither budget. Against a queueing gateway that holds
the request for the full timeout before answering 429, a call could hang
for 301 attempts / 25.2 hours, measured, versus 301 seconds before the
change.
The split is now by which budget the time consumes: time that burns
max_attempts is excluded from stall, time that does not (429 attempts
included) belongs to stall. Measured again: back to one attempt / 301s.
Only the chat loop needs this — embedding and ocr count 429 against
max_attempts unconditionally, so the gap never existed there. The stall
verdict moved into _stalled(), which both call sites had duplicated, to
keep __call__ under the complexity gate.
The concurrency case used two RetryMW instances, so instance-level sharing
was hidden by object isolation and a clock promoted to an instance
attribute passed all seven cases. Both cases now reuse one mw, and a new
one idles past the window between two calls on that instance — the shape
that would expose _entered_at pinned to process start. Mutation-checked:
promoting the clock fails the new case.
ARCHITECTURE.md 7.3 now carries the new metering and notes that the G6
ttft guard became conservative redundancy. The changelog entry leads with
what downstream must act on: the worst-case call duration rises to
max_attempts * timeout_s, and any STALL_WINDOW_S that was inflated to work
around this can go back to the default.
_validate_stall still guards stall_window_s >= max ttft_timeout_s, but its
stated reason no longer holds: TTFT waiting is productive time and never
reaches the stall account. The check is harmless and stays, so the
docstring now says why it is kept rather than implying a live hazard.
.env.example dropped the "must be >= max TTFT" advice for what the window
actually measures.
Same failure path as the embedding loop: one timed-out attempt drains the
wall-clock window, and the next round without a runnable source declares
the scope dead in _on_no_runnable. All three governance loops now meter
stall the same way.
The embedding loop shares the wall-clock entered_at and the same stall
verdict, so it failed the same way through a different path: one timed-out
attempt, then any round with no runnable source, and _on_no_runnable
declared the scope dead. Issue #8 only recorded the chat path; the
regression test pins this one.
Issue #8: with timeout_s >= stall_window_s a single timed-out request
exhausted the stall window before the second attempt was even dispatched,
so LLM_MAX_RETRIES never applied and the whole scope was declared dead.
Root cause is that real attempts and non-productive waiting charged the
same wall clock, while the stall budget is the smaller of the two. The new
StallClock subtracts attempt time from the stall account, leaving the two
budgets orthogonal: attempts bill max_attempts, waiting bills
stall_window_s. The dual-condition verdict, the inf semantics of
progress_age_s, the 429 exemption and the error surface are untouched.
The productive boundary is _attempt itself, telemetry included, so a slow
recorder cannot push a call into a stalled verdict.
Six tasks: StallClock plus the chat loop, then embedding, ocr, the config
comments, the full-suite regression with doc sync, and independent
verification. Codex review raised four points, all confirmed and folded in:
a stale line reference in the fidelity section, explicit cancellation
acceptance for T2/T3 (the new attempting() wrapper now wraps their existing
cancel paths), a telemetry-boundary test pinning the design's claim that
telemetry jitter must not feed the stall verdict, and concrete test
construction for the embedding/ocr regressions.
Two internal-consistency fixes from the independent design review:
the 429 saturation argument wrongly claimed exponential backoff growth
(429 skips the retry budget, so max(fails, 1) pins the delay to the base
tier), and "productive" was defined as waiting on the response while the
StallClock actually wraps all of _attempt. The boundary is now stated as
_attempt itself, including per-attempt accounting and telemetry, with the
rationale that telemetry jitter must not participate in the stall verdict.
Issue #8: a single request that burns its full timeout_s also exhausts
stall_window_s, so the retry budget silently never applies. Root cause is
that both budgets charge the same wall-clock time. The design makes the two
budgets orthogonal — real attempts bill the retry budget, everything else
bills the stall budget — which drops the timeout_s / stall_window_s coupling
instead of guarding it with an assembly-time check.
A fail-closed limiter or breaker backend means the scope cannot emit a
single request, yet GovernanceBackendError sat directly under
PolyGatewayError. A caller writing only `except GatewayUnavailableError`
dropped it into the catch-all branch, so a Redis blip burned a backlog's
business failure budget into the dead letter queue over a fault a restart
would clear. It now inherits GatewayUnavailableError with a
governance_backend_down reason and a 5 second retry_after_s.
The two unknown-source sites split out into SourceNotConfiguredError,
deliberately outside the retryable family: a misconfigured source name
must burn its budget and surface rather than retry forever in silence.
README now states which errors reach callers and which the retry loop
absorbs. TransientError and SourceDeadError read like caller contracts but
never arrive, and a downstream project wrote a whole design section on
that false premise before checking the source.
Independent verification caught the split not actually holding on the only
path production uses, and caught the fix for that opening a second hole on
the accounting path. Both are fixed and pinned by tests that go through
the wrappers rather than the private methods underneath them.
Letting SourceNotConfiguredError through the gate wrappers opened a hole
the recheck caught: _record_quietly only degrades GovernanceBackendError,
so an assembly defect raised from the accounting side would now escape and
destroy a response from a call that had already genuinely succeeded. That
inverts the exact invariant _record_quietly exists to hold.
Widening _record_quietly is the right fix rather than narrowing the
wrappers, because that layer degrades by what the path is (accounting, the
call is already done) rather than by which error type shows up. Narrowing
would have left 4 of 9 wrapper methods as exceptions to a rule nobody can
remember.
No backend raises it from an accounting method today, so this is a
guardrail for whoever adds source-name validation to a breaker backend.
The stub that first reported this green was wrong: its record_success
lacked count_attempt, so it raised TypeError and the wrapper relabeled it.
Fixed signature, then the test failed as it should have.
Also finishes the three-to-five leak path correction across the four
remaining spots, including the wiki summary card that indexes this design.
Independent verification caught that the split shipped in the previous
commit did not actually hold on the only path production uses. The gate
wrappers re-raise GovernanceBackendError but nothing else, so
SourceNotConfiguredError fell into the following `except Exception` and
came back out as a governance_backend_down failure with retry_after_s=5.0.
A misconfigured source name would still retry forever and never surface.
The existing tests missed it because both of them call the private _cfg()
directly, one layer below the wrapper the governance loops actually go
through. The regression test goes through QuotaGate.
telemetry.py has to widen its terminal catch in the same commit: once the
wrapper stops relabeling the error, it is no longer a GovernanceBackendError,
and it is raised before any attempt exists, so the path would have recorded
no telemetry at all.
Also corrects the leak path count from three to five. QuotaGate.stats and
BreakerGate.retry_after_s are not wrapped by _record_quietly either.
Minor rather than major: adding a parent class widens what an existing
`except` catches, it does not break one. Callers already catching
GovernanceBackendError keep working untouched.
The wiki sync in the release checklist is a no-op this time. The doc site
was taken down entirely on 2026-08-02 for accuracy reasons, and its
remaining landing page points at CHANGELOG.md as the version source of
truth, which this commit updates.
TransientError and SourceDeadError read like caller-facing contracts in
the taxonomy table, but the retry loop catches both and repackages them as
AllSourcesExhausted, so they never arrive. That is only discoverable by
reading middleware/retry.py, and a downstream project wrote a whole design
section on the false premise before checking.
The new table states the split outright, including that
GovernanceBackendError now sits on the caller-facing side and
SourceNotConfiguredError deliberately does not join the retryable family.
A fail-closed limiter or breaker backend means the scope cannot emit a
single request, which is exactly scope-level unavailability. But the error
sat directly under PolyGatewayError, so a caller writing only
`except GatewayUnavailableError` dropped it into the catch-all branch:
Redis blips once and a backlog of tasks burns its business failure budget
into the dead letter queue, over a fault a restart would clear.
Three gate paths leak to callers rather than being absorbed by
_record_quietly (try_acquire, try_enter, progress_age_s); each is now
pinned by a test, since none of them had one before.
The two unknown-source sites move to SourceNotConfiguredError instead of
following along. They report a misconfigured source name, not an outage,
and letting them into the retryable family would be the mirror of the bug
being fixed here: the task would retry forever and never surface.
Pure addition ahead of the reparenting, so this commit leaves every
existing caller and test untouched.
SourceNotConfiguredError deliberately stays outside GatewayUnavailableError:
a source name that is not in the limiter's config dict is an assembly
defect, not a transient outage, and folding it into the retryable family
would let a typo retry forever without ever reaching a dead letter queue.
The retry_after_s default is 5.0 rather than 0 because a backlog released
at zero delay would stampede a backend that is already down.
GovernanceBackendError arrived with the M2 distributed backends but never
made it into the section 6.1 table, so it had no place in the taxonomy
callers actually read. That omission is why the README missed it too.
Records the reparenting, the new governance_backend_down reason, and why
SourceNotConfiguredError deliberately stays outside the reparented family:
a misconfigured source name must burn its failure budget and surface,
not retry forever in silence.
Five tasks, with the ARCHITECTURE section 6.1 revision first so the code
never contradicts the single source of truth, and the reparenting kept
atomic because scope is a required keyword argument and any split would
leave an unrunnable tree.
The Codex review caught that the planned test evidence pointed at the
wrong stubs: the ones at test_backpressure.py:176-186 cover accounting-side
degradation, not the three gate paths that actually leak to callers, and
try_acquire and try_enter have no stub at all.
All three open decisions were settled as proposed: a dedicated
SourceNotConfiguredError so a misconfigured source reaches the dead
letter queue instead of retrying forever, a 5 second retry_after_s so a
backlog does not stampede a backend that is already down, and public
export so callers can alarm on assembly defects specifically.
Background dispatch produced two failure modes this session, and both
looked identical from the outside: a run that had finished without anyone
noticing, and a run that had wedged without anyone noticing. A pipe on the
end of the command masked pytest's real exit code as 0, and a waiter
looping on `pgrep -f "<the command>"` matched its own command line and
never terminated. Foreground execution trades parallelism for knowing
what actually happened.
Pins the ARCHITECTURE section 6.1 revision to land before or with the
implementation, since the new scope reason contradicts the current single
source of truth. Documents why SourceNotConfiguredError may sit outside
the four-way classification: that rule governs transport-translated call
failures, and the GatewayUnavailableError family already lives outside it.
Also collapses the ten per-field response ternaries in emit_attempt into
an _AttemptUsage view. They all expressed the same decision and pushed the
method to cyclomatic complexity C, which blocked the commit gate.
Fail-closed governance backend failures are semantically scope-level
unavailability, yet GovernanceBackendError sits directly under
PolyGatewayError, so callers writing only `except GatewayUnavailableError`
drop them into the catch-all bucket and burn their failure budget on a
fault that a restart would clear.
The design reparents it under GatewayUnavailableError with a new
governance_backend_down reason, splits the two "unknown source" sites into
a separate assembly-defect error so a misconfiguration still reaches the
dead letter queue, and picks a non-zero retry_after_s to avoid a
zero-delay retry storm against a backend that is already down.
The design claimed merging would immediately break dissect. It would
not: dissect keeps running whatever version it already has, and this
release does not touch it. What is true is narrower -- once dissect
moves to 1.0.6, the M2.7 scope will refuse to assemble.
Worth recording because the distinction is not academic here.
dissect/requirements.txt declares polygateway>=1.0.1,<1.1, a range
rather than a pin, so 1.0.6 satisfies it and any routine reinstall picks
it up without anyone deciding to upgrade. So it is not "breaks on
merge", it is "breaks on the next dependency install".
The paragraph now carries both corrections it went through, since a
claim about downstream impact that was wrong twice is worth leaving
visible rather than quietly rewriting.
enable_thinking=False was a silent no-op for minimax and openai sources.
The shape of the switch now stays at provider level while a model-level
capability table says whether a given model can honour it at all, and
reasoning_tokens is collected so the cost of thinking can be told apart
from the cost of answering.
Verified against the live gateway: a seventeen-row matrix over 137 real
calls, kept out of the CI gate behind the slow marker.
The thinking matrix had been running inside make ci all along, which is
not what the design claimed. It takes seven minutes, spends 137 real
calls, and its criteria are statistical, so a network hiccup fails the
build for reasons unrelated to the change under test -- one run died on
three consecutive network errors exhausting the source.
The project already has the mechanism for this: the slow marker, which
addopts excludes by default and the config comments describe as "CI runs
it on demand". Marking the matrix slow brings make ci back down from
seven minutes to ninety seconds while the matrix stays a merge
requirement via -m slow.
The design also claimed e2e does not run in CI. It does: make test runs
pytest over tests/, e2e included, and the existing smoke tests really
call the gateway whenever .env has credentials. Only slow-marked tests
are excluded. Both documents now say so.
Version sources are pyproject and __init__; a test enforces they agree,
and it caught the second one being missed.
The verifier caught that the disable-direction evidence only proved "no
regression", not "actually took effect": on M3 the disabled runs and the
no-opinion baseline are identically distributed, because that model does
not reason by default anyway. So the disable runs alone cannot rule out
the very failure mode issue #5 is about -- the parameter being silently
dropped upstream. The bogus-value experiment that does rule it out was
sitting in the findings document instead of the test suite; it is now
case L3b, and the L3 assertion that could never fail is gone.
Also from the review: the e2e helper caught bare Exception, which would
have disguised a library bug as an unavailable source, exactly the
silence the reporting discipline exists to prevent; the unregistered
model warning fired on every request instead of once per source; and the
transport caught ValueError broadly enough to mislabel unrelated errors,
now narrowed to a dedicated ThinkingUnsupportedError.
The design and plan still described the original judgement criteria,
which the measurements had already overturned. Both now match what the
tests actually do, and the design no longer claims the only new failure
surface is the openai one -- dissect configures MiniMax-M2.7 with
ENABLE_THINKING=false and will fail at assembly, which has to be
coordinated before this merges.
A sixteen-row matrix over 127 real calls: disable and enable on
MiniMax-M3 in both streaming and non-streaming mode, extra_body winning
over the profile slot, qwen and deepseek still disabling correctly, a
drift sentinel that re-derives every registered capability from live
behaviour, and the assembly guard refusing the models that cannot
comply.
Two judgement criteria had to be corrected by the data they were meant
to judge. Output length cannot separate the two regimes at all -- the
disabled runs reach 46 tokens when the model narrates its working in
the visible answer, and the enabled runs drop to 13 when medium effort
barely thinks. reasoning_tokens separates them cleanly in both
directions, which is precisely what issue #6 was collected for. A
second anchor compares prompt_tokens between the two regimes: the
vendor injects a reasoning instruction when thinking is on, so the
input side grows, and comparing the two runs relatively avoids
hardcoding any vendor number.
Provider names are mapped explicitly rather than guessed from the model
string; guessing had silently skipped the qwen row behind a "source
unavailable" reason that was not true.
enable_thinking=False was a no-op for minimax and openai sources: both
profiles had empty dicts on each side, so the payload update injected
nothing while the caller believed reasoning had been turned off. A
downstream project was blocked on exactly this.
The root cause is that an empty dict meant two different things -- "no
injection needed" and "we do not know how this provider spells it" --
and that a provider-level table cannot express what turned out to be a
per-model property. Live testing showed MiniMax-M3 can disable
reasoning via reasoning_effort while M2.7 and M2.5 cannot be disabled
at all, which two external registries independently confirm.
So the shape stays at provider level and a capability table joins it at
model level. Unknown, unsupported and no-opinion are now three distinct
values, and resolve_thinking is the single place they meet: it raises at
assembly time when a model cannot honour the request, warns and injects
for unregistered models, and injects silently otherwise. Every registered
capability carries the evidence it was derived from.
enable_thinking also joins the cache fingerprint, since it now really
does change the request body.
Reasoning tokens are already counted inside completion_tokens, so the
cost total was never wrong -- what was missing is the attribution: how
much of a call was spent thinking rather than answering.
LLMResponse and TransportResult each gain a trailing reasoning_tokens
field, and the telemetry port grows from 21 to 22 columns with the new
column appended in both backends so fresh and migrated schemas keep the
same physical order.
None means this particular call did not report the field, not that the
source never reports it: a relay that falls back to a local tokenizer
replaces the whole usage object and drops completion_tokens_details.
Downstream checks must therefore read "in (None, 0)"; no provider was
observed reporting a literal zero.
Ten tasks in a fixed order: land reasoning_tokens first so it can serve
as the acceptance instrument for the thinking-switch fix, then reshape
the provider profile, add the model-level capability table, wire the
assembly guard, fold enable_thinking into the cache fingerprint, and
verify the whole thing against the live API.
Incorporates a read-only Codex review: resolve_thinking now takes the
model name so its errors can name it, and the warning assertion uses a
loguru sink because caplog cannot see loguru output.
Findings: live-API measurements across MiniMax M3/M2.7/M2.5, qwen and
deepseek, plus a survey of how nine unified gateways model per-model
parameter divergence. Key facts: reasoning_effort is MiniMax's real
switch, M2.x reasoning is mandatory and cannot be disabled, and the
relay's local token-count fallback silently drops reasoning_tokens.
Design: keep the parameter shape at provider level, push capability
down to model level, split "unknown" / "unsupported" / "no opinion"
into three distinct values, and fail at assembly time when a model
cannot honour enable_thinking=False.
Both Redis backends have lowercased scope in their own constructors since
v1.0.0, so case never split the keyspace. What actually does is whitespace:
the backends lower but do not strip.
The test passed overlay and sampling as separate objects while production
aliases them, so an in-place mutation slipped through it. Also syncs the
telemetry schema page and adds the missing postgres round-trip assertion.
iomgaa