Commit Graph

5 Commits

Author SHA1 Message Date
iomgaa 45073486a7 fix: reparent governance backend failures under GatewayUnavailableError (issue #7)
A fail-closed limiter or breaker backend means the scope cannot emit a
single request, which is exactly scope-level unavailability. But the error
sat directly under PolyGatewayError, so a caller writing only
`except GatewayUnavailableError` dropped it into the catch-all branch:
Redis blips once and a backlog of tasks burns its business failure budget
into the dead letter queue, over a fault a restart would clear.

Three gate paths leak to callers rather than being absorbed by
_record_quietly (try_acquire, try_enter, progress_age_s); each is now
pinned by a test, since none of them had one before.

The two unknown-source sites move to SourceNotConfiguredError instead of
following along. They report a misconfigured source name, not an outage,
and letting them into the retryable family would be the mirror of the bug
being fixed here: the task would retry forever and never surface.
2026-08-06 04:53:52 -04:00
iomgaa 121888a0eb feat: suppress consecutive-channel opening on evidently healthy sources
Round 8 forensics caught the healthy source circuit-opened by five
random empty completions (~20% ambient failure rate makes a 5-streak
land every ~3000 attempts), blacking out the only good source for 60s.
When the window holds min_calls samples below the failure-rate
threshold, a streak is noise and no longer opens the gate; cold-start
and low-traffic semantics are unchanged and sudden death of a warm
source is still caught by the rate channel.
2026-07-21 15:28:34 -04:00
iomgaa e1678e4ff5 style: format M2.5 breaker changes 2026-07-21 09:11:56 -04:00
iomgaa c7ccb5798c feat: add failure-rate breaker channel with exponential reopen backoff
Dual-channel opening: consecutive failures (CHS-compatible, no streak
bump) plus windowed failure rate (two 30s buckets, min_calls guard).
429s bypass both channels as backpressure, and a probe hitting 429
releases instead of holding the lease. Open duration doubles per
rate/probe reopen up to max_cooldown_s, decaying after stable CLOSED.
record_success gains count_attempt so bad-result successes stay out of
the window.
2026-07-21 09:08:14 -04:00
iomgaa 733f413b08 feat: add redis circuit breaker gate with epoch fencing 2026-07-21 00:27:16 -04:00