Give one logical call an optional hard wall-clock boundary (issue #22).
Leaving it unset keeps 1.3.5 behaviour verbatim: the timeout context is
never entered when deadline_s is None.
- new deadline.py: ensure_call_deadline() range check (None or a finite
positive number; bool/0/nan/inf and out-of-range ints are rejected as
ValueError so OverflowError never leaks) plus with_call_deadline(),
which distinguishes an expiry from a TimeoutError raised by the body
or its cleanup via a local-variable identity comparison rather than
cm.expired() alone
- new CallDeadlineExceeded: deliberately outside the four categories and
not a GatewayUnavailableError, and carries no retry_after_s
- new {SCOPE}__CALL_DEADLINE_S key, guarded on the env, direct
construction and dataclasses.replace paths
- three clients take a call_deadline_s constructor argument and a
keyword-only per-call override on chat/embed/recognize_text/
parse_layout; None inherits the assembled value
- validation runs before the awaitable is created, so an illegal value
cannot strand an un-awaited coroutine
- one embed call shares a single deadline across all of its batches
- import-linter gains a polygateway.deadline layer
- cover where the deadline lands: backoff sleep, admission polling,
the structured re-ask ladder and embedding's batch loop, plus the
empty-texts early return that stays outside it
- cover what an expiry costs: exactly one terminal_failure row carrying
error_type=CallDeadlineExceeded, a cancelled attempt row sharing its
logical_call_id, cleanup that outlives the deadline (lower bound only)
and an already-billed success being discarded
- pin the injected clock as orthogonal: a 10^6 second jump never expires
a call, yet total_latency_ms still reads that clock
Grow the telemetry contract from 26 to 36 fields and give every logical
call a failure terminal row, so SQL can finally answer "how many calls
failed" and "why did the whole pool die".
Schema and port move together with the emitter writes in one commit:
splitting them would ship columns that nothing populates.
- schema: append 10 nullable columns (scope, operation, logical_call_id,
event_kind, http_status_code, error_type, cause_type, error_body,
attempts, total_latency_ms) to all five definition sites in one order
- ports: 10 keyword-only parameters without defaults; the protocol
signature is now the single source the assembly gate derives from
- emitter: take domain exception objects instead of pre-flattened text
and pin down the diagnostics in one helper; a relabelled 503 stays
503 and success rows leave all five columns NULL
- emitter: reject recorders whose record_llm_call cannot accept the
current field shape at assembly time, since _record would otherwise
swallow the TypeError and drop every row while calls keep succeeding
- clients: write at most one terminal row per logical call through a
single shared exit, deduplicated by the call context; TelemetryMW
stops writing terminals so the two sites cannot double count
- clients: cancellation stays best effort and propagates, non-domain
exceptions get no terminal row and keep their classification
- transports: give _status_to_error an explicit operation and fix the
historically mislabelled embedding HTTP failures
- structured: promote the bounded error formatter so the reask feedback
and the terminal explanation share one set of limits
Terminal rows carry no cost and no tokens, so cost aggregation is
unchanged; failure counts must now filter on event_kind.
Both public assembly paths took the tier on trust: a bare "none" from
JSON or a hand-built SourceConfig stayed a str, and `is Effort.NONE`
then read it as a contradiction and crashed on `.value` while wording
the error -- the caller got an AttributeError where a ValueError was
promised, and on the request side that unclassified exception walked
straight through the transport's ThinkingUnsupportedError catch and the
retry classifier.
Normalize at the two entrances instead, matching what the .env path has
always done, and let EFFORT_FALLBACK be spelled with the same freedom as
its neighbour.
The per-call reasoning tier never reached the cache key, and the model
fingerprint could not stand in for it: the fingerprint is computed once at
assembly time, so two calls on the same client asking for low and max looked
identical to it. Same messages, different tiers, one shared entry -- the
verbatim replay of issue #4's five seeds all hitting the same response.
Source-level tiers join the fingerprint under the same rule enable_thinking
already follows (appended only when the source takes a position), and the
filter that decides which sources enter the mark set is widened to match --
without that, a source configured with nothing but REASONING_EFFORT would
never reach _fingerprint_mark at all.
None (no opinion) and Effort.NONE (asked not to reason) stay distinct keys.
Sources that opine on neither keep byte-identical keys and fingerprints, so
nothing existing cold-starts.
The three-layer priority (call > source > enable_thinking sugar > silence)
now lives in one pure function, thinking.effective_effort(). The assembly
guard and the request hot path used to each carry their own inline copy of
the sugar conversion; two copies of the same judgement drift into the worst
shape there is - passes at assembly, raises at runtime.
The guard now also honours effort_fallback, so a source that opted into
nearest is no longer sentenced at assembly for a tier it could have mapped.
resolve_thinking now takes an Effort instead of a tri-state bool, and the
four gates become five. The new one sits ahead of the generic tier check
on purpose: asking for `none` on GLM-5.3 used to fall through to "none is
not supported, pick low/high/max", which loses both the fact that the
model cannot stop reasoning and the one tier the caller could switch to
right now. Without that alternative, downstream goes looking for
extra_body — which is how issue #20 happened in the first place.
The return type is a ThinkingResolution rather than the payload alone.
Under fallback="nearest" the tier that goes out is not the tier that was
asked for, and telemetry has to record the one that ran, or task 10 files
a call under a tier it never used. Ties in that mapping go to the weaker
side: a silent medium -> max is a multiple of the bill, and the library
does not raise a caller's price on its own.
Two readings the design left implicit, both settled the way its own
compatibility promise requires:
- `auto` is exempt from the tier list. It means "on, no tier named",
which in the body is the absence of the effort key, not a value of it.
Checking it against the list would break every existing source that
sets ENABLE_THINKING=true against deepseek-v4 or glm-5.3.
- `none` is never a mapping target. Turning "think less" into "do not
think" reverses the decision instead of cheapening it; a switch-only
model maps to `auto` and a model that only has `none` still errors.
Both call sites convert enable_thinking in place for now; task 5 folds
that into effective_effort along with the source- and call-level tiers.
providers.py had been holding two jobs: the registry of what each
provider looks like, and the decisions made from those declarations.
Adding response-side judgement would have made it the module for
everything about reasoning, so the decisions move to thinking.py and
the registry keeps only profiles and their lookup.
Moving a module breaks any deep-path import of what moved, so the six
public symbols are promoted to the package root at the same time. The
top level is this library's stated API surface; giving downstream a
stable name to import is what makes the next reorganisation harmless.
observe_thinking stays unexported — downstream reads the verdict off
LLMResponse, and exporting it would be a permanent promise for nothing.
The pool was the only external resource in the library that pre-allocated:
asyncpg's default min_size=10 turned pool creation into an all-or-nothing
action, so on a shared instance running low on connection budget the first
thing to fall over was the one component that must not fail silently
(4 clients x 10 = 40 idle connections just to write telemetry).
min_size=0 means "do not pre-connect" - asyncpg only builds holders - so
pool creation becomes free and never touches the database; connection
failures then land on acquire, the path that already drops one row and lets
the pool recover. max_size and the write budget become the library's
explicit statement about its own footprint, configurable through two new
keys whose defaults live in config alone (the recorder parameters are
required keyword-only, same discipline as auto_migrate).
The whole write - prepare, acquire, execute - now runs inside one
asyncio.timeout: acquire used to have no timeout at all, so a full pool
would hang forever on the caller's path. Release is explicit rather than
`async with`, because asyncpg shields release and reuses the acquire
timeout, which would let a single telemetry write consume twice the budget.
Telemetry degradation used to be a single warning and a private boolean.
In a long-running process that is indistinguishable from telemetry working:
issue #15 was only found by hand-reconciling milestone log lines against
llm_calls rows, after 19 calls had silently gone unrecorded. The SQLite
side was worse — once init failed, every write returned without even a
log line.
Degradation now has one shared owner. TelemetryStatusTracker holds the
state machine (enter/recover/drop/should-retry), announces entry and
recovery once each, and repeats the drop count under a row-and-time
double threshold so a degraded backend neither floods the log nor goes
quiet. Both recorders hold one; both count the rows they drop.
For programmatic consumers, TelemetryStatus is a frozen snapshot exposed
as telemetry_status on all three clients, resolved through a single
isinstance check. It is a separate optional port rather than a member of
TelemetryRecorder: that protocol is @runtime_checkable, so adding an
attribute would make every implementation that only defines
record_llm_call stop satisfying it — downstream isinstance assertions
would break on upgrade. The existing assertion in test_ports.py is what
keeps that decision honest.
Failure criteria are deliberately untouched here: Postgres still treats a
pool failure as permanent, only now visibly. `_failed` and the tracker
therefore both carry the verdict for the span of this one change; the
cooldown rework collapses them into the tracker alone.
A client used to close whatever transport, recorder or cache it happened
to hold, injected or not, so the first client to shut down killed the
backend its siblings were still using. That is why the explicit-sharing
path the architecture prescribes was unusable in practice and downstream
projects fell back to one private instance per client. The mirror image
of the same gap: the redis clients the factories build for the limiter
and the breaker were never closed at all, because nobody kept a
reference to them once they were handed to the retry middleware.
Ownership is now stated once, the way RedisLimiter already stated it:
whoever builds a resource closes it, injected ones are left alone. The
constructor is the full-injection path, so it owns nothing by default
and only the factories mark what they built. RedisCache gains the same
rule for its own client, and the three copies of the "probe for aclose,
fall back to close" dance collapse into a single helper so the next
correction cannot land in only one of them.
The telemetry write budget needs asyncio.timeout, whose uncancel accounting
was only fixed after 3.11.1 — pinning the floor at 3.12 removes that hazard
instead of working around it.
Raising ruff's target-version turns on UP047, so gather_bounded,
_anext_within and stream_with_liveness_timeouts move to def f[T](...) and
the two module-level TypeVars go away. That syntax is a SyntaxError on
3.11, so it can only land together with the version bump.
Limiter rejections have always chosen between waiting and failing fast;
breaker rejections had no such choice. The new key is the missing cell
of that matrix, shaped exactly like QUOTA_FULL so there is nothing new
to learn. It defaults to fail_fast: flipping the default would move
every existing deployment's worst-case wall clock from milliseconds to
the stall window, which is the wrong direction to impose on anyone.
Single-source scopes are the ones that want wait, and they now have a
way to say so.
The two keys stay separate despite sharing a domain, because a full
quota is "queue for your share" (your turn always comes) while an open
circuit is "wait for the source to recover" (it might not).
Policy validation collapses into SourceAdmission, the only consumer.
The three client constructors used to each carry their own copy of the
quota_full check; adding a second key there would have made eight
copies of the same two lines. Rejection timing and message are
unchanged -- admission is built inside those constructors.
This commit only wires the key through; the control flow that reads it
lands next.
`PGW_TELEMETRY_TEXT_CAP` now reaches the emitter on every assembly path.
Unset means no truncation, which stays the default: a truncated row is
no longer audit evidence and cannot be replayed, and downstreams rely on
that today. The flip side — contracts and bids sitting in `llm_calls`
indefinitely, multi-tenant — is spelled out in `.env.example` so readers
can weigh both.
All three `from_settings` paths are wired (chat, embedding, OCR): they
write the same table, so capping only chat would leave half of it
uncontrolled. `TelemetryEmitter.__init__` now rejects `text_cap <= 0`;
it is the single point where the three clients converge, so the direct
construction path — a public assembly route the settings guard never
sees — is covered too. `0` would otherwise reduce every body to a bare
elision marker.
Chat rows stored full message and response text with no upper bound, so
downstream contracts and tenders lived in llm_calls indefinitely. Add
_cap_text/_cap_messages in the single telemetry exit (_record), applied
after digest_messages and before json.dumps, plus to response/thinking.
Capping is per text, not over the serialized JSON: cutting the whole
string would emit invalid JSON into an unvalidated TEXT column. The cap
builds new dicts and never mutates in place — digest_messages passes
non-list content straight through as the same object, so an in-place cut
would silently poison the caller's messages and the cache key.
text_cap is required on TelemetryEmitter (internal class, three known
construction sites) and defaults to None on the three public clients, so
the default behaviour stays byte-for-byte identical. Settings wiring
lands separately.
Validation runs before the request enters the onion: every failure inside it
is downgraded to a warning by the telemetry layer, so validating in there
would not validate anything.
The dimensions stay out of the cache key — cache_namespace already carries
tenant isolation, and folding meta in would cold-start every existing entry.
enable_thinking=False was a no-op for minimax and openai sources: both
profiles had empty dicts on each side, so the payload update injected
nothing while the caller believed reasoning had been turned off. A
downstream project was blocked on exactly this.
The root cause is that an empty dict meant two different things -- "no
injection needed" and "we do not know how this provider spells it" --
and that a provider-level table cannot express what turned out to be a
per-model property. Live testing showed MiniMax-M3 can disable
reasoning via reasoning_effort while M2.7 and M2.5 cannot be disabled
at all, which two external registries independently confirm.
So the shape stays at provider level and a capability table joins it at
model level. Unknown, unsupported and no-opinion are now three distinct
values, and resolve_thinking is the single place they meet: it raises at
assembly time when a model cannot honour the request, warns and injects
for unregistered models, and injects silently otherwise. Every registered
capability carries the evidence it was derived from.
enable_thinking also joins the cache fingerprint, since it now really
does change the request body.
Round two of the from_env-only validation problem. Fifteen checks still lived
in the env parsing functions: six enum domains, the redis_url requirement for
redis-backed limiter/breaker/cache, cache namespace and TTL, telemetry path and
DSN, non-negative structured retries and non-blank scope. from_settings and
direct construction bypassed all of them.
The five asserts in client.py that claimed config had already validated
redis_url and the telemetry targets now hold on every path, so they revert to
what CLAUDE.md permits: internal invariant declarations that also narrow the
Optional for type checkers. Their comments now name the method that guarantees
them, since the previous wording is exactly what went stale.
Postgres DSNs built by hand now get the SQLAlchemy +driver suffix stripped the
way from_env has always stripped it, with a warning so the rewrite is not
silent. The env path strips earlier, so it stays quiet.
AIMD ceiling now respects per-source max_concurrency and the pacer is
assembled explicitly in the client; MIN_CALLS parses as strict int;
acceptance doc corrects source-5 attempt count to 549; design and
migration notes aligned with implemented 429/stall/suppression
semantics and AIMD constants documented.
Score is success-rate EWMA over (1 + inflight) with a 0.05 exploration
floor so quarantined sources can prove recovery; EWMA climb doubles as
slow-start. New optional OutcomeAwareSelector port feeds attempt
outcomes.
Classify empty completions as transient per human ruling (fixes flaky
real-gateway smoke and prevents caching empty responses), rename the
factory injection parameter gate to breaker per the frozen design,
rewrite the probe-entry cleanup without except BaseException, declare
python-dotenv explicitly, add a mid-backoff cancellation test, and
record all implementation errata in the design and architecture docs.
Includes config aggregation for multi-source env keys, from_env and
from_settings factories with explicit shared-backend injection,
gather_bounded, top-level exports, tightened import-linter layers with
the gate removed from the Makefile, and the finalized .env.example.
iomgaa