A client used to close whatever transport, recorder or cache it happened
to hold, injected or not, so the first client to shut down killed the
backend its siblings were still using. That is why the explicit-sharing
path the architecture prescribes was unusable in practice and downstream
projects fell back to one private instance per client. The mirror image
of the same gap: the redis clients the factories build for the limiter
and the breaker were never closed at all, because nobody kept a
reference to them once they were handed to the retry middleware.
Ownership is now stated once, the way RedisLimiter already stated it:
whoever builds a resource closes it, injected ones are left alone. The
constructor is the full-injection path, so it owns nothing by default
and only the factories mark what they built. RedisCache gains the same
rule for its own client, and the three copies of the "probe for aclose,
fall back to close" dance collapse into a single helper so the next
correction cannot land in only one of them.
retry_after_s never had a written definition, so each backend improvised
and they drifted apart. It now answers exactly one question: how long
until a retry is *certainly* worth attempting. OPEN has such a moment
(the cooldown deadline); HALF_OPEN does not, because the probe can come
back at any time -- so it reports 0.0, which already means "retry now"
elsewhere in the library.
Six exits are brought in line. The half-open rejection is the one issue
14 reported: it returned the probe lease remainder, a deadlock-guard
value derived from 2x the slowest timeout, so a 60s cooldown told
callers to wait 600s. Worse, retry.py fed that number into the source
cooldown memo, whose set_until only moves forward -- a source stayed
skipped in-process for the whole lease even after its probe succeeded
and the gate closed. That now writes an already-expired deadline, so
the memo goes back to recording only real OPEN cooldowns.
The other five were pre-existing memory/redis divergences hidden by a
contract-test blind spot (the suite pinned that a second caller gets
rejected, never what number it got): redis reported the probe TTL on
grant and the lease remainder on fenced-out writes, where memory has
always reported 0. Contract cases now pin all four half-open exits on
both backends, with 1:1 real-wait variants for redis since the
fake-clock ones skip there.
Not related to the schema work. These three fail ruff format --check on
main as well -- the pinned ruff is newer than whatever last formatted
them -- and a red make check makes the per-task quality gate useless for
everything that follows.
A fail-closed limiter or breaker backend means the scope cannot emit a
single request, which is exactly scope-level unavailability. But the error
sat directly under PolyGatewayError, so a caller writing only
`except GatewayUnavailableError` dropped it into the catch-all branch:
Redis blips once and a backlog of tasks burns its business failure budget
into the dead letter queue, over a fault a restart would clear.
Three gate paths leak to callers rather than being absorbed by
_record_quietly (try_acquire, try_enter, progress_age_s); each is now
pinned by a test, since none of them had one before.
The two unknown-source sites move to SourceNotConfiguredError instead of
following along. They report a misconfigured source name, not an outage,
and letting them into the retryable family would be the mirror of the bug
being fixed here: the task would retry forever and never surface.
Round 8 forensics caught the healthy source circuit-opened by five
random empty completions (~20% ambient failure rate makes a 5-streak
land every ~3000 attempts), blacking out the only good source for 60s.
When the window holds min_calls samples below the failure-rate
threshold, a streak is noise and no longer opens the gate; cold-start
and low-traffic semantics are unchanged and sudden death of a warm
source is still caught by the rate channel.
Dual-channel opening: consecutive failures (CHS-compatible, no streak
bump) plus windowed failure rate (two 30s buckets, min_calls guard).
429s bypass both channels as backpressure, and a probe hitting 429
releases instead of holding the lease. Open duration doubles per
rate/probe reopen up to max_cooldown_s, decaying after stable CLOSED.
record_success gains count_attempt so bad-result successes stay out of
the window.
Classify empty completions as transient per human ruling (fixes flaky
real-gateway smoke and prevents caching empty responses), rename the
factory injection parameter gate to breaker per the frozen design,
rewrite the probe-entry cleanup without except BaseException, declare
python-dotenv explicitly, add a mid-backoff cancellation test, and
record all implementation errata in the design and architecture docs.
Includes config aggregation for multi-source env keys, from_env and
from_settings factories with explicit shared-backend injection,
gather_bounded, top-level exports, tightened import-linter layers with
the gate removed from the Makefile, and the finalized .env.example.
iomgaa