providers.py had been holding two jobs: the registry of what each
provider looks like, and the decisions made from those declarations.
Adding response-side judgement would have made it the module for
everything about reasoning, so the decisions move to thinking.py and
the registry keeps only profiles and their lookup.
Moving a module breaks any deep-path import of what moved, so the six
public symbols are promoted to the package root at the same time. The
top level is this library's stated API surface; giving downstream a
stable name to import is what makes the next reorganisation harmless.
observe_thinking stays unexported — downstream reads the verdict off
LLMResponse, and exporting it would be a permanent promise for nothing.
The pool was the only external resource in the library that pre-allocated:
asyncpg's default min_size=10 turned pool creation into an all-or-nothing
action, so on a shared instance running low on connection budget the first
thing to fall over was the one component that must not fail silently
(4 clients x 10 = 40 idle connections just to write telemetry).
min_size=0 means "do not pre-connect" - asyncpg only builds holders - so
pool creation becomes free and never touches the database; connection
failures then land on acquire, the path that already drops one row and lets
the pool recover. max_size and the write budget become the library's
explicit statement about its own footprint, configurable through two new
keys whose defaults live in config alone (the recorder parameters are
required keyword-only, same discipline as auto_migrate).
The whole write - prepare, acquire, execute - now runs inside one
asyncio.timeout: acquire used to have no timeout at all, so a full pool
would hang forever on the caller's path. Release is explicit rather than
`async with`, because asyncpg shields release and reuses the acquire
timeout, which would let a single telemetry write consume twice the budget.
Telemetry degradation used to be a single warning and a private boolean.
In a long-running process that is indistinguishable from telemetry working:
issue #15 was only found by hand-reconciling milestone log lines against
llm_calls rows, after 19 calls had silently gone unrecorded. The SQLite
side was worse — once init failed, every write returned without even a
log line.
Degradation now has one shared owner. TelemetryStatusTracker holds the
state machine (enter/recover/drop/should-retry), announces entry and
recovery once each, and repeats the drop count under a row-and-time
double threshold so a degraded backend neither floods the log nor goes
quiet. Both recorders hold one; both count the rows they drop.
For programmatic consumers, TelemetryStatus is a frozen snapshot exposed
as telemetry_status on all three clients, resolved through a single
isinstance check. It is a separate optional port rather than a member of
TelemetryRecorder: that protocol is @runtime_checkable, so adding an
attribute would make every implementation that only defines
record_llm_call stop satisfying it — downstream isinstance assertions
would break on upgrade. The existing assertion in test_ports.py is what
keeps that decision honest.
Failure criteria are deliberately untouched here: Postgres still treats a
pool failure as permanent, only now visibly. `_failed` and the tracker
therefore both carry the verdict for the span of this one change; the
cooldown rework collapses them into the tracker alone.
A client used to close whatever transport, recorder or cache it happened
to hold, injected or not, so the first client to shut down killed the
backend its siblings were still using. That is why the explicit-sharing
path the architecture prescribes was unusable in practice and downstream
projects fell back to one private instance per client. The mirror image
of the same gap: the redis clients the factories build for the limiter
and the breaker were never closed at all, because nobody kept a
reference to them once they were handed to the retry middleware.
Ownership is now stated once, the way RedisLimiter already stated it:
whoever builds a resource closes it, injected ones are left alone. The
constructor is the full-injection path, so it owns nothing by default
and only the factories mark what they built. RedisCache gains the same
rule for its own client, and the three copies of the "probe for aclose,
fall back to close" dance collapse into a single helper so the next
correction cannot land in only one of them.
The telemetry write budget needs asyncio.timeout, whose uncancel accounting
was only fixed after 3.11.1 — pinning the floor at 3.12 removes that hazard
instead of working around it.
Raising ruff's target-version turns on UP047, so gather_bounded,
_anext_within and stream_with_liveness_timeouts move to def f[T](...) and
the two module-level TypeVars go away. That syntax is a SyntaxError on
3.11, so it can only land together with the version bump.
Limiter rejections have always chosen between waiting and failing fast;
breaker rejections had no such choice. The new key is the missing cell
of that matrix, shaped exactly like QUOTA_FULL so there is nothing new
to learn. It defaults to fail_fast: flipping the default would move
every existing deployment's worst-case wall clock from milliseconds to
the stall window, which is the wrong direction to impose on anyone.
Single-source scopes are the ones that want wait, and they now have a
way to say so.
The two keys stay separate despite sharing a domain, because a full
quota is "queue for your share" (your turn always comes) while an open
circuit is "wait for the source to recover" (it might not).
Policy validation collapses into SourceAdmission, the only consumer.
The three client constructors used to each carry their own copy of the
quota_full check; adding a second key there would have made eight
copies of the same two lines. Rejection timing and message are
unchanged -- admission is built inside those constructors.
This commit only wires the key through; the control flow that reads it
lands next.
`PGW_TELEMETRY_TEXT_CAP` now reaches the emitter on every assembly path.
Unset means no truncation, which stays the default: a truncated row is
no longer audit evidence and cannot be replayed, and downstreams rely on
that today. The flip side — contracts and bids sitting in `llm_calls`
indefinitely, multi-tenant — is spelled out in `.env.example` so readers
can weigh both.
All three `from_settings` paths are wired (chat, embedding, OCR): they
write the same table, so capping only chat would leave half of it
uncontrolled. `TelemetryEmitter.__init__` now rejects `text_cap <= 0`;
it is the single point where the three clients converge, so the direct
construction path — a public assembly route the settings guard never
sees — is covered too. `0` would otherwise reduce every body to a bare
elision marker.
Chat rows stored full message and response text with no upper bound, so
downstream contracts and tenders lived in llm_calls indefinitely. Add
_cap_text/_cap_messages in the single telemetry exit (_record), applied
after digest_messages and before json.dumps, plus to response/thinking.
Capping is per text, not over the serialized JSON: cutting the whole
string would emit invalid JSON into an unvalidated TEXT column. The cap
builds new dicts and never mutates in place — digest_messages passes
non-list content straight through as the same object, so an in-place cut
would silently poison the caller's messages and the cache key.
text_cap is required on TelemetryEmitter (internal class, three known
construction sites) and defaults to None on the three public clients, so
the default behaviour stays byte-for-byte identical. Settings wiring
lands separately.
Validation runs before the request enters the onion: every failure inside it
is downgraded to a warning by the telemetry layer, so validating in there
would not validate anything.
The dimensions stay out of the cache key — cache_namespace already carries
tenant isolation, and folding meta in would cold-start every existing entry.
enable_thinking=False was a no-op for minimax and openai sources: both
profiles had empty dicts on each side, so the payload update injected
nothing while the caller believed reasoning had been turned off. A
downstream project was blocked on exactly this.
The root cause is that an empty dict meant two different things -- "no
injection needed" and "we do not know how this provider spells it" --
and that a provider-level table cannot express what turned out to be a
per-model property. Live testing showed MiniMax-M3 can disable
reasoning via reasoning_effort while M2.7 and M2.5 cannot be disabled
at all, which two external registries independently confirm.
So the shape stays at provider level and a capability table joins it at
model level. Unknown, unsupported and no-opinion are now three distinct
values, and resolve_thinking is the single place they meet: it raises at
assembly time when a model cannot honour the request, warns and injects
for unregistered models, and injects silently otherwise. Every registered
capability carries the evidence it was derived from.
enable_thinking also joins the cache fingerprint, since it now really
does change the request body.
Round two of the from_env-only validation problem. Fifteen checks still lived
in the env parsing functions: six enum domains, the redis_url requirement for
redis-backed limiter/breaker/cache, cache namespace and TTL, telemetry path and
DSN, non-negative structured retries and non-blank scope. from_settings and
direct construction bypassed all of them.
The five asserts in client.py that claimed config had already validated
redis_url and the telemetry targets now hold on every path, so they revert to
what CLAUDE.md permits: internal invariant declarations that also narrow the
Optional for type checkers. Their comments now name the method that guarantees
them, since the previous wording is exactly what went stale.
Postgres DSNs built by hand now get the SQLAlchemy +driver suffix stripped the
way from_env has always stripped it, with a warning so the rewrite is not
silent. The env path strips earlier, so it stays quiet.
AIMD ceiling now respects per-source max_concurrency and the pacer is
assembled explicitly in the client; MIN_CALLS parses as strict int;
acceptance doc corrects source-5 attempt count to 549; design and
migration notes aligned with implemented 429/stall/suppression
semantics and AIMD constants documented.
Score is success-rate EWMA over (1 + inflight) with a 0.05 exploration
floor so quarantined sources can prove recovery; EWMA climb doubles as
slow-start. New optional OutcomeAwareSelector port feeds attempt
outcomes.
Classify empty completions as transient per human ruling (fixes flaky
real-gateway smoke and prevents caching empty responses), rename the
factory injection parameter gate to breaker per the frozen design,
rewrite the probe-entry cleanup without except BaseException, declare
python-dotenv explicitly, add a mid-backoff cancellation test, and
record all implementation errata in the design and architecture docs.
Includes config aggregation for multi-source env keys, from_env and
from_settings factories with explicit shared-backend injection,
gather_bounded, top-level exports, tightened import-linter layers with
the gate removed from the Makefile, and the finalized .env.example.
iomgaa