Eight tasks against the approved design, ordered so the port and both
telemetry backends land before the three call paths that feed them.
Writing the plan turned up a third telemetry path the design missed:
OcrClient emits through the same helper and builds its ChatRequest on
the spot, just as embedding does. OCR rows share the table with chat
rows, so leaving them out would put a hole in a multi-tenant caller's
audit trail, and the same irreversibility argument applies. Listed as
Task 6 and flagged as beyond the approved scope -- it may be dropped,
but only by stating the limitation in the CHANGELOG, not silently.
The integration task pins the issue's own argument as a test: build a
22-column table, open it with the current recorder, and assert the old
rows read back as the empty string rather than NULL -- NULL under an
RLS policy is invisible to everyone, not merely unassigned.
The embedding client does not go through the chat onion -- embed() runs
its own chain down to _emit(), which builds a ChatRequest on the spot
and so far only fills session_id and parent_call_id. Changing chat()
alone would have left every embed row with empty dimensions, which is
exactly what the issue's second request asks for.
The bigger find: the draft claimed serialization could not fail because
the entry check already restricts values to scalars. It can. A float
passes a naive type check and json.dumps writes it as the literal NaN,
which is not valid JSON and which JSONB rejects; the failure then lands
in the emitter's degrade path and turns a caller's input error into
silently dropped telemetry. Now rejected at the entry with isfinite and
again at serialization with allow_nan=False.
Also states the validation runs at both public entries, not just chat(),
and adds the RLS template the design had promised but never wrote down.
Issue #11 asks for a tenant column so a multi-tenant caller can isolate
rows in the database. Widened to caller-defined dimensions in general,
but only the caller's own: model name and friends keep their existing
columns, and the library writes nothing into the new container.
Two independent findings force tenant_id to be a real column rather than
a key inside JSON. An RLS policy on meta->>'tenant_id' parses fine, but
the planner discards statistics for non-LEAKPROOF functions under RLS,
and ->> is not marked leakproof; the pgsql-general report that hit this
ended up moving the indexed column out of JSONB. Separately, the planner
has no usable statistics for JSONB at all -- @> falls back to a
hardcoded 0.1% selectivity.
A configurable promoted-column whitelist is rejected: when two
downstreams infer different types for the same key, the second
ADD COLUMN is silently skipped by IF NOT EXISTS and the wrong type is
written from then on, without an error.
The library stops at the column plus a documented policy template. It
must never enable RLS itself -- with no matching policy that is
default-deny, which would silently fail every write for the two
downstreams that are not multi-tenant.
Measured 201 on POST /api/v1/packages/iomgaa/pypi/polygateway/-/link/
PolyGateway during the 1.2.0 release. The note saying it 404s and must
be done through the web UI would have sent the next release down a
manual path that is no longer needed.
Verifier mutation test: flipping _translate_429 to parse the summary
left all 824 tests green. The padding was one long string value, so the
cut landed inside it - and head-and-tail retention kept the trailing
error object, leaving the summary parseable. Many keys put the cut
between structural tokens, where the summary stops being valid JSON.
Mutation now fails as it should. Also splits OCR 429 out on its own.
Issue #10 Task 6. The install pin moves from ==1.1.* to >=1.2,<2 - left
alone, everyone following the README would have stayed silently on
1.1.2 without this fix and without a warning. Telemetry field count
re-measured via inspect.signature: still 22.
Issue #10 Task 5, the acceptance claim. Before the fix this asserted
against 'qwen_1 请求被拒: 400' and failed on the first substring - which
is exactly what the downstream batch was left with. Uses the real body
from the issue, and checks the trailing code too, since a head-only cut
would drop the one field you quote when chasing the provider.
Issue #10 Task 4: the OCR side said only 'HTTP 404'. The issue reported
the chat path, but the batch that lost its 400 was reading tables - the
same blind spot, one transport over. Reuses the shared summarizer.
Issue #10 Task 3: five branches each built their own message, so adding
the summary would have meant five copies. Table-driven classification
composes it in one place instead, and the 429 split still parses the
untruncated body - reading the summary would demote an oversized
insufficient_quota to a plain rate limit and stop force_open.
Issue #10 Task 2: head-and-tail rather than a head-only cut, because the
code and request_id that let you chase the provider sit at the very end
of a JSON error body. Cap 2048 follows k8s client-go for the same job.
Issue #10 Task 1: a rejected call's reason had nowhere to live. The
field goes on the base class because these errors all come from one HTTP
response - which class it is and what the peer said are orthogonal.
Codex review: the code blocks reference httpx and PolyGatewayError, but
neither module imports them today. A zero-context implementer copying
them verbatim would stall on F821.
The 500-char head-only rule came from a single sample. k8s client-go
caps the same thing at 2048; reprlib keeps head and tail because the
text is meant to be read. Gateway error bodies are JSON whose code and
request_id sit at the very end, so a head-only cut drops exactly what
you need to chase the provider. Version pinned at 1.2.0, which forces
the README install pin off ==1.1.*.
"Truncate at cap and append the ellipsis" admits both 501 and 500 total
length; the two would desync test assertions from the telemetry length
promise. Cap is now the total including the marker.
Issue #10: the 400 body dies in _status_to_error, and telemetry only
writes str(exc), so adding a field alone would not make the refusal
queryable after the fact. Design keeps the summary in both the message
and a new base-class body_text, across every non-2xx branch and both
transports.
1.1.2 went out with an empty description on the registry page: without a
readme field there is no long_description, and twine only warns about
that -- it does not block the upload. Add readme and project.urls, and
record the wider lesson in the release procedure: a release is done when
the pages a downstream user actually opens look right, not when the local
steps go green. Also adds the missing step for creating a Release, which
is why the Releases page sat empty through eight tags.
README first, per the release procedure: the 1.1.* pin still covers this
version, the 22-field count re-checked with inspect.signature, and the
telemetry row now states that an existing table needs no schema CREATE
privilege.
PostgreSQL checks the schema CREATE privilege before the IF NOT EXISTS
existence test, so an account with only table-level INSERT was denied on
CREATE TABLE IF NOT EXISTS even though the table was right there and
writable. The denial set _failed and the whole recorder went no-op for
the process lifetime, silently: 150+ calls downstream lost their latency,
token and cost rows with nothing but one warning to show for it.
The probe is the direct fix. The larger fix is the criterion: structural
degradation now means "provably cannot write" (pool creation failed, or
the table is absent and cannot be created), not "something threw during
init" -- a probe or acquire failure just skips the row and retries on the
next call.
SQLite stays as it is on purpose. Measured: it short-circuits the
statement at parse time, so it passes even under another connection's
EXCLUSIVE lock or on a read-only file. A probe there would buy nothing;
the docstring now says so to keep symmetry-minded future edits away.
Bumping the version is not releasing. 1.0.6 and 1.1.0 both got a version
bump and a changelog entry but were never uploaded, so the registry sat at
1.0.5 and downstream could not install any of those fixes.
The ordering matters in one non-obvious way: README has to be correct
before the build, because sdist freezes whatever is there at that moment.
That is exactly how 1.1.1 shipped with a stale README. The install pin is
called out by name since it is the easiest line to forget and the most
damaging to leave wrong.
Also records where the Gitea token actually lives — tea's config, not
.pypirc — after that misreading led to a wrong "no credentials" claim.
Three README drifts, none of them about issue #8 alone:
- the telemetry row said 18 fields; record_llm_call takes 22 (verified by
inspect.signature). ports.py claimed 20 in its own docstring, so both
records of the same fact were stale.
- LLMResponse.cost is hardcoded to None on the chat path (retry.py:519).
Cost only ever reaches telemetry. The old doc site named this as a known
trap, so the capability row now says it outright.
- backpressure had no row at all, which is what issue #8 was about.
The pin still said ==1.0.*, which caps downstream at 1.0.5 and hides
every fix since. Now that 1.1.1 is actually in the registry the pin can
move; it was missed when 1.1.0 was tagged.
Patch rather than minor: the error surface is unchanged and no public
signature moved. What downstream must notice is timing, not types — the
worst-case call duration rises to roughly max_attempts * timeout_s now
that the retry budget actually applies.
Pre-release review caught an overreaching promise in the changelog entry:
the 429 bound holds only when the stall verdict can fire at all, i.e. when
the whole scope has no progress. The verdict is a conjunction, so a call
does not die while other calls in the scope are still producing — by
design — which leaves no hard per-call ceiling in that case. That property
predates this fix and is now stated with its precondition instead of as an
unconditional guarantee.
The wiki sync in the release checklist is a no-op again: the doc site has
been down since 2026-08-02 and its landing page names CHANGELOG.md as the
version source of truth, which this commit updates.
Registration pages carry the chosen approach, why the split is by "which
budget the time consumes", the five rejected alternatives with reasons,
and the 3.6 correction found during independent verification.
Independent verification found the first cut had swapped one bug for a
worse one. The budgets were split by "did we send a request", so a 429
attempt counted as productive — but 429 is exempt from the retry budget,
so its time burned neither budget. Against a queueing gateway that holds
the request for the full timeout before answering 429, a call could hang
for 301 attempts / 25.2 hours, measured, versus 301 seconds before the
change.
The split is now by which budget the time consumes: time that burns
max_attempts is excluded from stall, time that does not (429 attempts
included) belongs to stall. Measured again: back to one attempt / 301s.
Only the chat loop needs this — embedding and ocr count 429 against
max_attempts unconditionally, so the gap never existed there. The stall
verdict moved into _stalled(), which both call sites had duplicated, to
keep __call__ under the complexity gate.
The concurrency case used two RetryMW instances, so instance-level sharing
was hidden by object isolation and a clock promoted to an instance
attribute passed all seven cases. Both cases now reuse one mw, and a new
one idles past the window between two calls on that instance — the shape
that would expose _entered_at pinned to process start. Mutation-checked:
promoting the clock fails the new case.
ARCHITECTURE.md 7.3 now carries the new metering and notes that the G6
ttft guard became conservative redundancy. The changelog entry leads with
what downstream must act on: the worst-case call duration rises to
max_attempts * timeout_s, and any STALL_WINDOW_S that was inflated to work
around this can go back to the default.
_validate_stall still guards stall_window_s >= max ttft_timeout_s, but its
stated reason no longer holds: TTFT waiting is productive time and never
reaches the stall account. The check is harmless and stays, so the
docstring now says why it is kept rather than implying a live hazard.
.env.example dropped the "must be >= max TTFT" advice for what the window
actually measures.
Same failure path as the embedding loop: one timed-out attempt drains the
wall-clock window, and the next round without a runnable source declares
the scope dead in _on_no_runnable. All three governance loops now meter
stall the same way.
The embedding loop shares the wall-clock entered_at and the same stall
verdict, so it failed the same way through a different path: one timed-out
attempt, then any round with no runnable source, and _on_no_runnable
declared the scope dead. Issue #8 only recorded the chat path; the
regression test pins this one.
Issue #8: with timeout_s >= stall_window_s a single timed-out request
exhausted the stall window before the second attempt was even dispatched,
so LLM_MAX_RETRIES never applied and the whole scope was declared dead.
Root cause is that real attempts and non-productive waiting charged the
same wall clock, while the stall budget is the smaller of the two. The new
StallClock subtracts attempt time from the stall account, leaving the two
budgets orthogonal: attempts bill max_attempts, waiting bills
stall_window_s. The dual-condition verdict, the inf semantics of
progress_age_s, the 429 exemption and the error surface are untouched.
The productive boundary is _attempt itself, telemetry included, so a slow
recorder cannot push a call into a stalled verdict.
Six tasks: StallClock plus the chat loop, then embedding, ocr, the config
comments, the full-suite regression with doc sync, and independent
verification. Codex review raised four points, all confirmed and folded in:
a stale line reference in the fidelity section, explicit cancellation
acceptance for T2/T3 (the new attempting() wrapper now wraps their existing
cancel paths), a telemetry-boundary test pinning the design's claim that
telemetry jitter must not feed the stall verdict, and concrete test
construction for the embedding/ocr regressions.
Two internal-consistency fixes from the independent design review:
the 429 saturation argument wrongly claimed exponential backoff growth
(429 skips the retry budget, so max(fails, 1) pins the delay to the base
tier), and "productive" was defined as waiting on the response while the
StallClock actually wraps all of _attempt. The boundary is now stated as
_attempt itself, including per-attempt accounting and telemetry, with the
rationale that telemetry jitter must not participate in the stall verdict.
Issue #8: a single request that burns its full timeout_s also exhausts
stall_window_s, so the retry budget silently never applies. Root cause is
that both budgets charge the same wall-clock time. The design makes the two
budgets orthogonal — real attempts bill the retry budget, everything else
bills the stall budget — which drops the timeout_s / stall_window_s coupling
instead of guarding it with an assembly-time check.
A fail-closed limiter or breaker backend means the scope cannot emit a
single request, yet GovernanceBackendError sat directly under
PolyGatewayError. A caller writing only `except GatewayUnavailableError`
dropped it into the catch-all branch, so a Redis blip burned a backlog's
business failure budget into the dead letter queue over a fault a restart
would clear. It now inherits GatewayUnavailableError with a
governance_backend_down reason and a 5 second retry_after_s.
The two unknown-source sites split out into SourceNotConfiguredError,
deliberately outside the retryable family: a misconfigured source name
must burn its budget and surface rather than retry forever in silence.
README now states which errors reach callers and which the retry loop
absorbs. TransientError and SourceDeadError read like caller contracts but
never arrive, and a downstream project wrote a whole design section on
that false premise before checking the source.
Independent verification caught the split not actually holding on the only
path production uses, and caught the fix for that opening a second hole on
the accounting path. Both are fixed and pinned by tests that go through
the wrappers rather than the private methods underneath them.
Letting SourceNotConfiguredError through the gate wrappers opened a hole
the recheck caught: _record_quietly only degrades GovernanceBackendError,
so an assembly defect raised from the accounting side would now escape and
destroy a response from a call that had already genuinely succeeded. That
inverts the exact invariant _record_quietly exists to hold.
Widening _record_quietly is the right fix rather than narrowing the
wrappers, because that layer degrades by what the path is (accounting, the
call is already done) rather than by which error type shows up. Narrowing
would have left 4 of 9 wrapper methods as exceptions to a rule nobody can
remember.
No backend raises it from an accounting method today, so this is a
guardrail for whoever adds source-name validation to a breaker backend.
The stub that first reported this green was wrong: its record_success
lacked count_attempt, so it raised TypeError and the wrapper relabeled it.
Fixed signature, then the test failed as it should have.
Also finishes the three-to-five leak path correction across the four
remaining spots, including the wiki summary card that indexes this design.
Independent verification caught that the split shipped in the previous
commit did not actually hold on the only path production uses. The gate
wrappers re-raise GovernanceBackendError but nothing else, so
SourceNotConfiguredError fell into the following `except Exception` and
came back out as a governance_backend_down failure with retry_after_s=5.0.
A misconfigured source name would still retry forever and never surface.
The existing tests missed it because both of them call the private _cfg()
directly, one layer below the wrapper the governance loops actually go
through. The regression test goes through QuotaGate.
telemetry.py has to widen its terminal catch in the same commit: once the
wrapper stops relabeling the error, it is no longer a GovernanceBackendError,
and it is raised before any attempt exists, so the path would have recorded
no telemetry at all.
Also corrects the leak path count from three to five. QuotaGate.stats and
BreakerGate.retry_after_s are not wrapped by _record_quietly either.
Minor rather than major: adding a parent class widens what an existing
`except` catches, it does not break one. Callers already catching
GovernanceBackendError keep working untouched.
The wiki sync in the release checklist is a no-op this time. The doc site
was taken down entirely on 2026-08-02 for accuracy reasons, and its
remaining landing page points at CHANGELOG.md as the version source of
truth, which this commit updates.
TransientError and SourceDeadError read like caller-facing contracts in
the taxonomy table, but the retry loop catches both and repackages them as
AllSourcesExhausted, so they never arrive. That is only discoverable by
reading middleware/retry.py, and a downstream project wrote a whole design
section on the false premise before checking.
The new table states the split outright, including that
GovernanceBackendError now sits on the caller-facing side and
SourceNotConfiguredError deliberately does not join the retryable family.
A fail-closed limiter or breaker backend means the scope cannot emit a
single request, which is exactly scope-level unavailability. But the error
sat directly under PolyGatewayError, so a caller writing only
`except GatewayUnavailableError` dropped it into the catch-all branch:
Redis blips once and a backlog of tasks burns its business failure budget
into the dead letter queue, over a fault a restart would clear.
Three gate paths leak to callers rather than being absorbed by
_record_quietly (try_acquire, try_enter, progress_age_s); each is now
pinned by a test, since none of them had one before.
The two unknown-source sites move to SourceNotConfiguredError instead of
following along. They report a misconfigured source name, not an outage,
and letting them into the retryable family would be the mirror of the bug
being fixed here: the task would retry forever and never surface.
Pure addition ahead of the reparenting, so this commit leaves every
existing caller and test untouched.
SourceNotConfiguredError deliberately stays outside GatewayUnavailableError:
a source name that is not in the limiter's config dict is an assembly
defect, not a transient outage, and folding it into the retryable family
would let a typo retry forever without ever reaching a dead letter queue.
The retry_after_s default is 5.0 rather than 0 because a backlog released
at zero delay would stampede a backend that is already down.
GovernanceBackendError arrived with the M2 distributed backends but never
made it into the section 6.1 table, so it had no place in the taxonomy
callers actually read. That omission is why the README missed it too.
Records the reparenting, the new governance_backend_down reason, and why
SourceNotConfiguredError deliberately stays outside the reparented family:
a misconfigured source name must burn its failure budget and surface,
not retry forever in silence.
Five tasks, with the ARCHITECTURE section 6.1 revision first so the code
never contradicts the single source of truth, and the reparenting kept
atomic because scope is a required keyword argument and any split would
leave an unrunnable tree.
The Codex review caught that the planned test evidence pointed at the
wrong stubs: the ones at test_backpressure.py:176-186 cover accounting-side
degradation, not the three gate paths that actually leak to callers, and
try_acquire and try_enter have no stub at all.
All three open decisions were settled as proposed: a dedicated
SourceNotConfiguredError so a misconfigured source reaches the dead
letter queue instead of retrying forever, a 5 second retry_after_s so a
backlog does not stampede a backend that is already down, and public
export so callers can alarm on assembly defects specifically.
iomgaa