Both transport paths now share one verdict: a stream is complete on
[DONE] or on finish_reason followed by measured usage and a clean EOF
(both modes; missing_done only governs real truncation). Outcomes follow
the design's first-match table: upstream_error finishes stay transient,
fidelity mode delivers reasoning-only/length results, text mode raises
ResultInvalid for content-less length/filter endings, and non-stream
fidelity judges before building the snapshot so truncated tools reach
tool_truncated. Projections honour the reasoning alias, dialect warnings
are bounded, and responses carry finish_reason and stream_end.
Red: 39 of 46 tests in tests/unit/test_completion_transport.py failed
before this change (e.g. eof_after_finish raised 保真响应缺 [DONE];
reasoning-only length raised empty_completion; reasoning-only stream hit
the TTFT watchdog, re-confirmed by disabling alias liveness).
Null is treated as absent at every layer; unknown delta, tool and
function fields merge by OpenAI streaming conventions (strings concat,
label keys idempotent, indexed arrays by index) and conflicts drop the
key and record its path instead of failing the call. The accumulator
now tracks finish and usage-after-finish in both modes, exposes the
reasoning alias, and non-stream parsing splits shape checks from
snapshot construction.
Red: 20 of 47 tests in tests/unit/test_openai_completion.py failed
before this change (e.g. test_null_known_and_unknown_fields_are_absent
raised TransientError 未知 delta 字段; finish_seen/raw_message missing).
Pure functions in types.py classify finish reasons, judge stream
completeness and outcome, and decide cacheability; TransportResult and
LLMResponse gain finish_reason and stream_end; snapshots accept
tool_calls null.
Red: tests/unit/test_completion_verdict.py failed at collection
(ImportError: _classify_finish et al. undefined) before this change.
Add faithful single-choice chat results, tool stream activity, error-frame retries, and isolated protocol caching.
Reject empty cached completions with the same value predicate as transport, add regression coverage, and adopt Apache-2.0.
HEDGE__STALL_AFTER_S triggers a cross-source hedge when the primary
leg stalls mid-stream (independent of HEDGE__AFTER_S); two-phase
hedge window with event recheck discipline; watchdog-killed primary
converges on the surviving hedge leg. STREAM__STALL_THRESHOLD_MS
(1000ms default) adjustable via env/settings/constructor. Assembly
guards: ValueError on never-opening windows, embedding/OCR scope
warning (B4). Batch D 14 tests red-to-green, 1688 unit+contracts.
Third approved port change: TelemetryRecorder.record_llm_call gains
stall_ms/stall_count (assembly gate derives from this signature).
Schema five edit points (both DDLs, both backfill decls, COLUMNS)
with INTEGER nullable columns appended last; telemetry_schema_sql
regenerates from the same constants untouched. Emitter forwards the
pair from retry._emit on the success branch only (cancelled hedge
loser rows stay NULL by construction); cache-hit and terminal rows
pass explicit None. Both **fields recorders pick the columns up with
zero signature change.
CallStats gains stall_ms/stall_count (None = no stall observation);
winner sample triple (wall, stall_ms, stall_count) converted once in
_attempt success branch; net = max(0, wall - stall) converged into the
shared _record_winner_sample used by both the single-path guard and
hedge winner registration. Two-fail hedge rounds record nothing
(no winner to attribute). Design section 4.2; issues #27.
Address branch review findings 1-4 on feature/1.3.7-hedged-requests:
- src/polygateway/middleware/retry.py: recheck primary.done() after
hedge admission in _attempt_hedged; release the hedge permit via
settle_and_release(permit, 0) (release_probe for probe entries) and
adjudicate the primary directly instead of firing a billable HTTP
request that would be cancelled immediately
- src/polygateway/config.py: check_hedge_assembly raises a hedge-located
ValueError for empty sources instead of a bare min() error
- tests/unit/test_hedge.py: pin that an injected FakeClock jump of 10^6
seconds does not trigger hedging (design section 8); pin pick(exclude)
counting no gate_rejections and leaving reasons untouched; pin silent
hedge abandonment when the candidate circuit is open; deterministic
regression for the admission-window race (BlockingLimiter harness)
- tests/unit/test_config.py: assert the empty-sources guard message
locates the hedge key
Red-to-green evidence in tests/outputs/137/review-fixes/
CallStats gains hedges/generation_ms/hedge_won (all defaulted, appended
after total_latency_ms); _CallContext counts them via record_generation
(overwrite for chat/OCR, accumulate for embedding batches) and
register_hedge, and snapshot carries them out. All three _attempt
implementations time only the transport call itself on the same injected
clock as total_latency_ms; the chat sink is recorded by the orchestrator
so hedge winner attribution stays with T3. Hedge counters stay 0/False
until the T3 orchestration lands.
Red-green evidence: tests/outputs/137/t2/ (10 new tests AttributeError
red, then green; full unit+contracts 1621 passed).
Transport.complete gains the keyword-only first_token_event (no default,
per the port convention): streaming sets it on the first delta, the
non-streaming path accepts it but never sets it, None means the caller
does not observe the first token. All fake/wrapping transports and the
three direct call sites follow the signature; the e2e wrapper forwards.
Red-green evidence: tests/outputs/137/t1/ (batch A TypeError red, then
147 file tests + 1550 unit tests green).
A gateway that answers 429 with Retry-After: inf (or 1e999, which float()
happily rounds to inf) used to reach backoff_delay as retry_after_s=inf.
max(delay, retry_after) then picked it, and since the library deliberately
does not clamp the hint with backoff_max_s, the attempt slept forever.
- _parse_retry_after now rejects non-finite values via math.isinf and
returns None, so the call falls back to plain exponential backoff
- it emits exactly one warning carrying the source name and the verdict
word retry_after_not_finite, never the raw header: under a 429 storm an
echoed header drowns the real signal and does not help localisation
- source_name becomes a required keyword-only argument; the function is
private, so no default is given and a missed call site fails loudly
instead of silently dropping the source identity from the warning
- nan keeps flowing through the existing seconds > 0 semantics; no new
branch, no reordering of the parse
Give one logical call an optional hard wall-clock boundary (issue #22).
Leaving it unset keeps 1.3.5 behaviour verbatim: the timeout context is
never entered when deadline_s is None.
- new deadline.py: ensure_call_deadline() range check (None or a finite
positive number; bool/0/nan/inf and out-of-range ints are rejected as
ValueError so OverflowError never leaks) plus with_call_deadline(),
which distinguishes an expiry from a TimeoutError raised by the body
or its cleanup via a local-variable identity comparison rather than
cm.expired() alone
- new CallDeadlineExceeded: deliberately outside the four categories and
not a GatewayUnavailableError, and carries no retry_after_s
- new {SCOPE}__CALL_DEADLINE_S key, guarded on the env, direct
construction and dataclasses.replace paths
- three clients take a call_deadline_s constructor argument and a
keyword-only per-call override on chat/embed/recognize_text/
parse_layout; None inherits the assembled value
- validation runs before the awaitable is created, so an illegal value
cannot strand an un-awaited coroutine
- one embed call shares a single deadline across all of its batches
- import-linter gains a polygateway.deadline layer
- cover where the deadline lands: backoff sleep, admission polling,
the structured re-ask ladder and embedding's batch loop, plus the
empty-texts early return that stays outside it
- cover what an expiry costs: exactly one terminal_failure row carrying
error_type=CallDeadlineExceeded, a cancelled attempt row sharing its
logical_call_id, cleanup that outlives the deadline (lower bound only)
and an already-billed success being discarded
- pin the injected clock as orthogonal: a 10^6 second jump never expires
a call, yet total_latency_ms still reads that clock
Grow the telemetry contract from 26 to 36 fields and give every logical
call a failure terminal row, so SQL can finally answer "how many calls
failed" and "why did the whole pool die".
Schema and port move together with the emitter writes in one commit:
splitting them would ship columns that nothing populates.
- schema: append 10 nullable columns (scope, operation, logical_call_id,
event_kind, http_status_code, error_type, cause_type, error_body,
attempts, total_latency_ms) to all five definition sites in one order
- ports: 10 keyword-only parameters without defaults; the protocol
signature is now the single source the assembly gate derives from
- emitter: take domain exception objects instead of pre-flattened text
and pin down the diagnostics in one helper; a relabelled 503 stays
503 and success rows leave all five columns NULL
- emitter: reject recorders whose record_llm_call cannot accept the
current field shape at assembly time, since _record would otherwise
swallow the TypeError and drop every row while calls keep succeeding
- clients: write at most one terminal row per logical call through a
single shared exit, deduplicated by the call context; TelemetryMW
stops writing terminals so the two sites cannot double count
- clients: cancellation stays best effort and propagates, non-domain
exceptions get no terminal row and keep their classification
- transports: give _status_to_error an explicit operation and fix the
historically mislabelled embedding HTTP failures
- structured: promote the bounded error formatter so the reask feedback
and the terminal explanation share one set of limits
Terminal rows carry no cost and no tokens, so cost aggregation is
unchanged; failure counts must now filter on event_kind.
The transport half of the effort_fallback wiring got a test last round; the
assembly half did not. Mutating _guard_thinking's fallback=source.effort_fallback
to a hardcoded "error" leaves the whole suite green, yet a zhipu/glm-5.3 source
carrying REASONING_EFFORT=medium + EFFORT_FALLBACK=nearest goes from assembling
fine to being refused at assembly. Pin it down: from_env must return a client.
The minimax wire lost its tier value on the assumption that these models
reason by default, so injecting nothing still reads as "on". T10 measured
the real gateway and the assumption does not hold: MiniMax-M3 with no
reasoning parameter did not reason in 5 of 5 rounds, while all six
strength values worked. Existing downstreams on ENABLE_THINKING=true
went from reasoning to silently not reasoning, and the capability table
cannot catch it because phase 5 lets auto through unconditionally.
Restore on_base to the old {"reasoning_effort": "medium"} verbatim. This
is a stopgap - it hands the tier choice back to the library, which this
work set out to remove. The real fix is to constrain auto by the
capability table, a public behaviour change tracked as issue #21.
The assertions that said "minimax injects no tier on the on-tier" go
back with it; each carries a note on why it moved twice.
`test_extra_body_overrides_the_profile_slot` guards that a source's
`extra_body` wins over the slot the profile injects. Since T5 made
`enable_thinking=True` mean `auto`, its minimax fixture injected an empty
fragment — swapping the two update calls changed nothing and the whole
suite stayed green, while a real source (zhipu, glm-5.3, tier `low`)
would have shipped `low` instead of the `high` the caller wrote. Move the
fixture onto that combination, where the two sides write the same key.
`emit_attempt`'s docstring promises `reasoning_applies` carries no
default so that a missed argument is a TypeError rather than a silent
`True`; nothing enforced it. Pin the signature the way `record_llm_call`
is already pinned.
`resolve_thinking` is exported, and its third argument turned from `bool`
into `Effort` this cycle — so the most natural downstream call now passes
the `"low"` it read out of JSON or config. Every gate inside compares by
identity, and `"none" is Effort.NONE` is always false: the tier was not
rejected, it was silently misjudged. Phase 2 read the on-form for a
request that wanted the off-form, Phase 4 was skipped entirely on a model
that cannot be disabled, and the only visible symptom arrived much later
as an AttributeError from `.value` — an exception that is neither
documented nor one of the four error classes.
Design 4.4 already lists this as the fourth entry a tier can come in
through; this makes the code agree with it.
Twenty-five columns and not one of them answered "which tier was this?",
so the question the whole issue exists to settle - does a higher tier buy
anything - had no way to group its data.
The three emit entry points deliberately disagree, the way sampling
already does. A successful attempt records what the transport actually
sent: with EFFORT_FALLBACK=nearest a request for medium goes out as low,
and recomputing here would file the row under a tier that never left the
process. A failed attempt has no response to read, so it falls back to
the requested tier - which is exactly right for the tier errors that are
rejected before any HTTP happens, because the rejected tier is the
signal. Cache hits and terminal failures have no chosen source at all,
so a source-level tier is not a thing they could report.
emit_attempt now demands to be told whether the path reasons at all.
Embedding and OCR share the emitter but never send reasoning parameters;
without the flag a source that mistakenly carries ENABLE_THINKING would
hang a tier on a call that could not possibly have run at one.
The value lands as a plain str. StrEnum is a str subclass and asyncpg
promises nothing about encoding subclasses, and a telemetry write that
fails is only a warning - Postgres would just quietly lose the column.
NULL means nobody declared a tier, which is not the same statement as
'none', and the two must never be folded together.
Adding applied_effort to LLMResponse put it through the cache round
trip, where JSON stores a StrEnum as its plain value. Rehydrated raw, a
hit would hand downstream a str while the annotation says Effort, and
every `is Effort.LOW` in the library would quietly answer False on the
hit path only -- the same trap thinking_observation already has a
coercion for.
A value outside this version's vocabulary degrades to None rather than
failing the entry: projects sharing one Redis would otherwise keep
invalidating each other's writes over an attribution field, and None is
the honest reading of a tier this version cannot name.
The transport now hands back the tier it actually sent, and that tier
rides TransportResult into LLMResponse. It is not the requested one:
under EFFORT_FALLBACK=nearest a medium request goes out as low, and
telemetry grouping by the requested tier would file the row under a tier
that never left the process.
Reconciliation judges the same tier instead of the old enable_thinking
bool, and the warning throttle keys on it. Keyed on the bool, every tier
of one model shared a single key, so the second contradiction was
silenced for the lifetime of the transport. The predicate is an identity
check against Effort.NONE on purpose -- the member's value is the
non-empty string "none", so any truthiness test would send every strength
tier down the "asked to disable" branch and invert the alarm.
The absent-tier side has had goldens since issue #4; the side that
actually carries a tier had none, so swapping str() for repr() in the
cache key changed the literal without turning the suite red. Pin both
literals. Note the fingerprint's json.dumps emits the same bytes for an
Effort member and its .value, so that pair is indistinguishable by any
test -- the golden pins the literal itself instead.
Both were wired but unwitnessed: hardcoding the transport's fallback, or
blanking the source tier the assembly guard reads, left the whole unit
suite green. Cover them where the value is visible -- the bytes on the
wire for nearest-vs-error, and the assembly-time refusal that must name
low as the executable alternative.
A provider that registered a disable form but no enable form was told
its shape was unknown and pointed at register_provider -- work it had
already done -- for a request that only ever needed the disable form.
The old bool code took the slot by direction; the tiered rewrite lost
that. Take the relevant field again, and keep "shape unknown" for the
case where both halves are missing, so the "cannot disable" wording
still owns the half-missing case.
Both public assembly paths took the tier on trust: a bare "none" from
JSON or a hand-built SourceConfig stayed a str, and `is Effort.NONE`
then read it as a contradiction and crashed on `.value` while wording
the error -- the caller got an AttributeError where a ValueError was
promised, and on the request side that unclassified exception walked
straight through the transport's ThinkingUnsupportedError catch and the
retry classifier.
Normalize at the two entrances instead, matching what the .env path has
always done, and let EFFORT_FALLBACK be spelled with the same freedom as
its neighbour.
The per-call reasoning tier never reached the cache key, and the model
fingerprint could not stand in for it: the fingerprint is computed once at
assembly time, so two calls on the same client asking for low and max looked
identical to it. Same messages, different tiers, one shared entry -- the
verbatim replay of issue #4's five seeds all hitting the same response.
Source-level tiers join the fingerprint under the same rule enable_thinking
already follows (appended only when the source takes a position), and the
filter that decides which sources enter the mark set is widened to match --
without that, a source configured with nothing but REASONING_EFFORT would
never reach _fingerprint_mark at all.
None (no opinion) and Effort.NONE (asked not to reason) stay distinct keys.
Sources that opine on neither keep byte-identical keys and fingerprints, so
nothing existing cold-starts.
The Transport port took the request apart into five arguments, so a tier
placed on ChatRequest could never reach _build_payload: the field was set,
read by nobody, and silently ignored - the exact shape of failure that sent
downstream to extra_body in the first place.
complete() now takes reasoning_effort with no default, matching the
TelemetryRecorder convention: a default would turn a missing hand-off into
a silent 'no opinion'. All four fakes move with it, since @runtime_checkable
checks method names and not signatures.
EmbeddingTransport and OcrTransport are deliberately left alone - they have
no reasoning semantics - and a test now holds that line.
_build_payload drops its inline sugar conversion for effective_effort(), so
the guard and the hot path share one judgement, and passes the source's
effort_fallback for the same reason.
The three-layer priority (call > source > enable_thinking sugar > silence)
now lives in one pure function, thinking.effective_effort(). The assembly
guard and the request hot path used to each carry their own inline copy of
the sugar conversion; two copies of the same judgement drift into the worst
shape there is - passes at assembly, raises at runtime.
The guard now also honours effort_fallback, so a source that opted into
nearest is no longer sentenced at assembly for a tier it could have mapped.
resolve_thinking now takes an Effort instead of a tri-state bool, and the
four gates become five. The new one sits ahead of the generic tier check
on purpose: asking for `none` on GLM-5.3 used to fall through to "none is
not supported, pick low/high/max", which loses both the fact that the
model cannot stop reasoning and the one tier the caller could switch to
right now. Without that alternative, downstream goes looking for
extra_body — which is how issue #20 happened in the first place.
The return type is a ThinkingResolution rather than the payload alone.
Under fallback="nearest" the tier that goes out is not the tier that was
asked for, and telemetry has to record the one that ran, or task 10 files
a call under a tier it never used. Ties in that mapping go to the weaker
side: a silent medium -> max is a multiple of the bill, and the library
does not raise a caller's price on its own.
Two readings the design left implicit, both settled the way its own
compatibility promise requires:
- `auto` is exempt from the tier list. It means "on, no tier named",
which in the body is the absence of the effort key, not a value of it.
Checking it against the list would break every existing source that
sets ENABLE_THINKING=true against deepseek-v4 or glm-5.3.
- `none` is never a mapping target. Turning "think less" into "do not
think" reverses the decision instead of cheapening it; a switch-only
model maps to `auto` and a model that only has `none` still errors.
Both call sites convert enable_thinking in place for now; task 5 folds
that into effective_effort along with the source- and call-level tiers.
The previous commit added the name to __all__ but never bound it, so
`from polygateway import ThinkingWire` and `import *` both raised while
the whole suite stayed green — the export test names symbols one by one,
and nobody thought to add the new one.
The guard is now the invariant rather than a longer list: every name in
__all__ must be an attribute of the package.
iomgaa