Unrecognizable bytes are rejected before any request. The result package is downloaded as a stream with a cumulative size cap, and error statuses are classified before the size check. Parsing runs in a worker thread so cancellation and call deadlines stay effective. parse_layout shares the first protocol stage without behavior change.
The service wraps every failure, including its own 400s, as a 500 with a parsing-failed detail. Only a fixed set of known bad-input markers is narrowed to a rejected request, and only when the caller passes markers; the shared default classification is unchanged.
Content comes from the middle json and its paired markdown; the content list is deliberately ignored because it escapes markdown characters. The external-data exception boundary covers both reading and structure parsing so no native exception escapes the OCR client. The layout parser is unchanged.
The service dispatches on the upload filename extension. A PDF sent as image.jpg made /ocr/text return a success response with hallucinated text. Only PDFs are relabelled; everything else keeps the legacy image label.
OcrDocumentElement extends the layout element with text; OcrDocumentResult stays independent because the parent's defaulted call_stats forbids a required markdown field. OcrDocumentTransport is a separate Protocol so existing OcrTransport implementers keep passing isinstance.
- fidelity cacheability also requires a non-empty projected body, so
think-tag-only answers are no longer cached
- a no-argument tool sent with arguments null only is a deterministic
dialect: ResultInvalid tool_arguments_null instead of retried
- text mode treats choice index null as 0 and no longer lets False pose
as 0
- null-only tool_calls elements no longer produce an empty tool_calls
list; non-stream fidelity checks role before the verdict; warning
paths are length-bounded
- docs: ARCHITECTURE 7.8 field counts, README cache/watchdog wording,
CHANGELOG notes, review record in the design, hook note in the plan
Red: 8 new tests failed before this change (e.g.
test_openai_cacheable_requires_projected_body,
test_null_arguments_tool_is_result_invalid_stream,
test_text_mode_choice_index_null_is_zero[None-True]).
llm_calls gains a nullable stream_end TEXT column (auto backfill,
manual-mode trimming as in 1.3.8). TelemetryRecorder.record_llm_call
takes a new required keyword stream_end; the emitter reads it from the
response on attempt rows and writes NULL for failed, cache-hit and
terminal rows.
Red: 18 tests in tests/unit/test_telemetry.py and test_ports.py failed
against the pre-change source (missing column, 38-field signature,
stream_end absent from emitted rows).
Cacheable is now a strict subset of deliverable: salvaged streams,
content-less results and length/filter/unknown finishes are delivered
but never written, and the read path applies the same verdict to the
stored values before hit overrides (invalid stream_end or non-string
finish read as a miss; 1.3.9 fidelity entries fall back to the
completion's finish). Hits reset stream_end. The legacy
_has_completion_output helper is folded into _is_cacheable.
Red: 13 tests in tests/unit/test_cache.py failed before this change
(uncacheable results were written and replayed; new fields not reset).
Both transport paths now share one verdict: a stream is complete on
[DONE] or on finish_reason followed by measured usage and a clean EOF
(both modes; missing_done only governs real truncation). Outcomes follow
the design's first-match table: upstream_error finishes stay transient,
fidelity mode delivers reasoning-only/length results, text mode raises
ResultInvalid for content-less length/filter endings, and non-stream
fidelity judges before building the snapshot so truncated tools reach
tool_truncated. Projections honour the reasoning alias, dialect warnings
are bounded, and responses carry finish_reason and stream_end.
Red: 39 of 46 tests in tests/unit/test_completion_transport.py failed
before this change (e.g. eof_after_finish raised 保真响应缺 [DONE];
reasoning-only length raised empty_completion; reasoning-only stream hit
the TTFT watchdog, re-confirmed by disabling alias liveness).
Null is treated as absent at every layer; unknown delta, tool and
function fields merge by OpenAI streaming conventions (strings concat,
label keys idempotent, indexed arrays by index) and conflicts drop the
key and record its path instead of failing the call. The accumulator
now tracks finish and usage-after-finish in both modes, exposes the
reasoning alias, and non-stream parsing splits shape checks from
snapshot construction.
Red: 20 of 47 tests in tests/unit/test_openai_completion.py failed
before this change (e.g. test_null_known_and_unknown_fields_are_absent
raised TransientError 未知 delta 字段; finish_seen/raw_message missing).
Pure functions in types.py classify finish reasons, judge stream
completeness and outcome, and decide cacheability; TransportResult and
LLMResponse gain finish_reason and stream_end; snapshots accept
tool_calls null.
Red: tests/unit/test_completion_verdict.py failed at collection
(ImportError: _classify_finish et al. undefined) before this change.
Add faithful single-choice chat results, tool stream activity, error-frame retries, and isolated protocol caching.
Reject empty cached completions with the same value predicate as transport, add regression coverage, and adopt Apache-2.0.
_minimal_fields and the pre_135 acceptance dicts missed the two new
columns, KeyErroring against migrated 39-column physical tables
(caught by the pre-merge integration gate; production path is
signature-gated and unaffected). Verifier gaps P2-1/P2-2 closed:
full suite 1764 passed, PG 30 passed, lint green.
HEDGE__STALL_AFTER_S triggers a cross-source hedge when the primary
leg stalls mid-stream (independent of HEDGE__AFTER_S); two-phase
hedge window with event recheck discipline; watchdog-killed primary
converges on the surviving hedge leg. STREAM__STALL_THRESHOLD_MS
(1000ms default) adjustable via env/settings/constructor. Assembly
guards: ValueError on never-opening windows, embedding/OCR scope
warning (B4). Batch D 14 tests red-to-green, 1688 unit+contracts.
Third approved port change: TelemetryRecorder.record_llm_call gains
stall_ms/stall_count (assembly gate derives from this signature).
Schema five edit points (both DDLs, both backfill decls, COLUMNS)
with INTEGER nullable columns appended last; telemetry_schema_sql
regenerates from the same constants untouched. Emitter forwards the
pair from retry._emit on the success branch only (cancelled hedge
loser rows stay NULL by construction); cache-hit and terminal rows
pass explicit None. Both **fields recorders pick the columns up with
zero signature change.
CallStats gains stall_ms/stall_count (None = no stall observation);
winner sample triple (wall, stall_ms, stall_count) converted once in
_attempt success branch; net = max(0, wall - stall) converged into the
shared _record_winner_sample used by both the single-path guard and
hedge winner registration. Two-fail hedge rounds record nothing
(no winner to attribute). Design section 4.2; issues #27.
Address branch review findings 1-4 on feature/1.3.7-hedged-requests:
- src/polygateway/middleware/retry.py: recheck primary.done() after
hedge admission in _attempt_hedged; release the hedge permit via
settle_and_release(permit, 0) (release_probe for probe entries) and
adjudicate the primary directly instead of firing a billable HTTP
request that would be cancelled immediately
- src/polygateway/config.py: check_hedge_assembly raises a hedge-located
ValueError for empty sources instead of a bare min() error
- tests/unit/test_hedge.py: pin that an injected FakeClock jump of 10^6
seconds does not trigger hedging (design section 8); pin pick(exclude)
counting no gate_rejections and leaving reasons untouched; pin silent
hedge abandonment when the candidate circuit is open; deterministic
regression for the admission-window race (BlockingLimiter harness)
- tests/unit/test_config.py: assert the empty-sources guard message
locates the hedge key
Red-to-green evidence in tests/outputs/137/review-fixes/
CallStats gains hedges/generation_ms/hedge_won (all defaulted, appended
after total_latency_ms); _CallContext counts them via record_generation
(overwrite for chat/OCR, accumulate for embedding batches) and
register_hedge, and snapshot carries them out. All three _attempt
implementations time only the transport call itself on the same injected
clock as total_latency_ms; the chat sink is recorded by the orchestrator
so hedge winner attribution stays with T3. Hedge counters stay 0/False
until the T3 orchestration lands.
Red-green evidence: tests/outputs/137/t2/ (10 new tests AttributeError
red, then green; full unit+contracts 1621 passed).
Transport.complete gains the keyword-only first_token_event (no default,
per the port convention): streaming sets it on the first delta, the
non-streaming path accepts it but never sets it, None means the caller
does not observe the first token. All fake/wrapping transports and the
three direct call sites follow the signature; the e2e wrapper forwards.
Red-green evidence: tests/outputs/137/t1/ (batch A TypeError red, then
147 file tests + 1550 unit tests green).
A gateway that answers 429 with Retry-After: inf (or 1e999, which float()
happily rounds to inf) used to reach backoff_delay as retry_after_s=inf.
max(delay, retry_after) then picked it, and since the library deliberately
does not clamp the hint with backoff_max_s, the attempt slept forever.
- _parse_retry_after now rejects non-finite values via math.isinf and
returns None, so the call falls back to plain exponential backoff
- it emits exactly one warning carrying the source name and the verdict
word retry_after_not_finite, never the raw header: under a 429 storm an
echoed header drowns the real signal and does not help localisation
- source_name becomes a required keyword-only argument; the function is
private, so no default is given and a missed call site fails loudly
instead of silently dropping the source identity from the warning
- nan keeps flowing through the existing seconds > 0 semantics; no new
branch, no reordering of the parse
Give one logical call an optional hard wall-clock boundary (issue #22).
Leaving it unset keeps 1.3.5 behaviour verbatim: the timeout context is
never entered when deadline_s is None.
- new deadline.py: ensure_call_deadline() range check (None or a finite
positive number; bool/0/nan/inf and out-of-range ints are rejected as
ValueError so OverflowError never leaks) plus with_call_deadline(),
which distinguishes an expiry from a TimeoutError raised by the body
or its cleanup via a local-variable identity comparison rather than
cm.expired() alone
- new CallDeadlineExceeded: deliberately outside the four categories and
not a GatewayUnavailableError, and carries no retry_after_s
- new {SCOPE}__CALL_DEADLINE_S key, guarded on the env, direct
construction and dataclasses.replace paths
- three clients take a call_deadline_s constructor argument and a
keyword-only per-call override on chat/embed/recognize_text/
parse_layout; None inherits the assembled value
- validation runs before the awaitable is created, so an illegal value
cannot strand an un-awaited coroutine
- one embed call shares a single deadline across all of its batches
- import-linter gains a polygateway.deadline layer
- cover where the deadline lands: backoff sleep, admission polling,
the structured re-ask ladder and embedding's batch loop, plus the
empty-texts early return that stays outside it
- cover what an expiry costs: exactly one terminal_failure row carrying
error_type=CallDeadlineExceeded, a cancelled attempt row sharing its
logical_call_id, cleanup that outlives the deadline (lower bound only)
and an already-billed success being discarded
- pin the injected clock as orthogonal: a 10^6 second jump never expires
a call, yet total_latency_ms still reads that clock
Migrate the PG telemetry fixtures to the 36-field recorder and add the
storage compatibility acceptance the plan calls for.
Mechanical migration:
- _EXPECTED_COLUMNS 27 -> 37 physical columns
- _record_minimal gains the ten keys in the same shape as the unit suite
- _PRE_TENANT_COLUMNS now excludes 14 columns, derived from
_CALL_OBSERVABILITY_COLUMNS instead of a second hand-written list, and
the two manual-mode warnings assert a notice derived from COLUMNS order
so a column that silently drops out of the warning turns the test red
New TestCallObservabilityColumnsAcceptance, all on a 27-column 1.3.4
shaped table built by the existing pg_sandbox factory:
- auto appends the ten columns in the same order as a fresh database and
old rows keep NULL in every one of them (no backfill, no sentinel)
- manual sends no DDL, trims the INSERT, and still round-trips the other
26 columns value by value
- an old-version writer using insert_sql with the 1.3.4 column set and a
new-version writer share one table, and event_kind filtering counts
neither the old rows as failures nor as successes
_minimal_fields is split out of _record_minimal so the simulated old
process reuses the same values rather than copying them.
Verified against the real lab Postgres: 30 passed.
Grow the telemetry contract from 26 to 36 fields and give every logical
call a failure terminal row, so SQL can finally answer "how many calls
failed" and "why did the whole pool die".
Schema and port move together with the emitter writes in one commit:
splitting them would ship columns that nothing populates.
- schema: append 10 nullable columns (scope, operation, logical_call_id,
event_kind, http_status_code, error_type, cause_type, error_body,
attempts, total_latency_ms) to all five definition sites in one order
- ports: 10 keyword-only parameters without defaults; the protocol
signature is now the single source the assembly gate derives from
- emitter: take domain exception objects instead of pre-flattened text
and pin down the diagnostics in one helper; a relabelled 503 stays
503 and success rows leave all five columns NULL
- emitter: reject recorders whose record_llm_call cannot accept the
current field shape at assembly time, since _record would otherwise
swallow the TypeError and drop every row while calls keep succeeding
- clients: write at most one terminal row per logical call through a
single shared exit, deduplicated by the call context; TelemetryMW
stops writing terminals so the two sites cannot double count
- clients: cancellation stays best effort and propagates, non-domain
exceptions get no terminal row and keep their classification
- transports: give _status_to_error an explicit operation and fix the
historically mislabelled embedding HTTP failures
- structured: promote the bounded error formatter so the reask feedback
and the terminal explanation share one set of limits
Terminal rows carry no cost and no tokens, so cost aggregation is
unchanged; failure counts must now filter on event_kind.
The 2:25 slow run left exactly one red: kimi-for-coding answers 404
model_not_found because the channel removed it from the account group
between 09:44 (four green probes, correct model_reported) and 15:00. L8
was reading that as "the capability table drifted", which is a statement
about the model the channel no longer serves.
The 404/model_not_found rule already used by T10 now lives in one helper
and is applied on the L1-L9 side too, via the same unreachable fallback:
that one rejection skips and records an uncovered row, every other
RequestRejectedError still bubbles, since those are the real failures
this suite exists to catch.
L9's "unknown shape" sample was the openai profile, which 1.3.3 gave a real
shape (off/on_base/effort_key all set), so the guard had nothing to reject.
It now registers a shapeless provider of its own and tests the mechanism
rather than whichever profile happens to be blank that month.
L8 checks the reported model before judging the capability table: this channel
answers glm-5 / glm-5.1 / glm-5.2 with glm-5.3, which is a routing problem the
library already warns about, not drift. All three are guarded, including the
one that passed by luck.
T10 tells 404 model_not_found (the channel dropped the model) apart from 400
(the tier really is refused), reading the status code and the body's type field
rather than the whole message; only the latter still counts as a conclusion
about a tier. An all-skipped tier list now skips instead of going green.
TestMiniMaxM3 gained the unreachable fallback its own docstring promised: an
outage now skips and leaves an uncovered row, where before it failed ahead of
_record and left no trace of what happened.
The transport half of the effort_fallback wiring got a test last round; the
assembly half did not. Mutating _guard_thinking's fallback=source.effort_fallback
to a hardcoded "error" leaves the whole suite green, yet a zhipu/glm-5.3 source
carrying REASONING_EFFORT=medium + EFFORT_FALLBACK=nearest goes from assembling
fine to being refused at assembly. Pin it down: from_env must return a client.
The minimax wire lost its tier value on the assumption that these models
reason by default, so injecting nothing still reads as "on". T10 measured
the real gateway and the assumption does not hold: MiniMax-M3 with no
reasoning parameter did not reason in 5 of 5 rounds, while all six
strength values worked. Existing downstreams on ENABLE_THINKING=true
went from reasoning to silently not reasoning, and the capability table
cannot catch it because phase 5 lets auto through unconditionally.
Restore on_base to the old {"reasoning_effort": "medium"} verbatim. This
is a stopgap - it hands the tier choice back to the library, which this
work set out to remove. The real fix is to constrain auto by the
capability table, a public behaviour change tracked as issue #21.
The assertions that said "minimax injects no tier on the on-tier" go
back with it; each carries a note on why it moved twice.
`test_extra_body_overrides_the_profile_slot` guards that a source's
`extra_body` wins over the slot the profile injects. Since T5 made
`enable_thinking=True` mean `auto`, its minimax fixture injected an empty
fragment — swapping the two update calls changed nothing and the whole
suite stayed green, while a real source (zhipu, glm-5.3, tier `low`)
would have shipped `low` instead of the `high` the caller wrote. Move the
fixture onto that combination, where the two sides write the same key.
`emit_attempt`'s docstring promises `reasoning_applies` carries no
default so that a missed argument is a TypeError rather than a silent
`True`; nothing enforced it. Pin the signature the way `record_llm_call`
is already pinned.
`resolve_thinking` is exported, and its third argument turned from `bool`
into `Effort` this cycle — so the most natural downstream call now passes
the `"low"` it read out of JSON or config. Every gate inside compares by
identity, and `"none" is Effort.NONE` is always false: the tier was not
rejected, it was silently misjudged. Phase 2 read the on-form for a
request that wanted the off-form, Phase 4 was skipped entirely on a model
that cannot be disabled, and the only visible symptom arrived much later
as an AttributeError from `.value` — an exception that is neither
documented nor one of the four error classes.
Design 4.4 already lists this as the fourth entry a tier can come in
through; this makes the code agree with it.
Twenty-five columns and not one of them answered "which tier was this?",
so the question the whole issue exists to settle - does a higher tier buy
anything - had no way to group its data.
The three emit entry points deliberately disagree, the way sampling
already does. A successful attempt records what the transport actually
sent: with EFFORT_FALLBACK=nearest a request for medium goes out as low,
and recomputing here would file the row under a tier that never left the
process. A failed attempt has no response to read, so it falls back to
the requested tier - which is exactly right for the tier errors that are
rejected before any HTTP happens, because the rejected tier is the
signal. Cache hits and terminal failures have no chosen source at all,
so a source-level tier is not a thing they could report.
emit_attempt now demands to be told whether the path reasons at all.
Embedding and OCR share the emitter but never send reasoning parameters;
without the flag a source that mistakenly carries ENABLE_THINKING would
hang a tier on a call that could not possibly have run at one.
The value lands as a plain str. StrEnum is a str subclass and asyncpg
promises nothing about encoding subclasses, and a telemetry write that
fails is only a warning - Postgres would just quietly lose the column.
NULL means nobody declared a tier, which is not the same statement as
'none', and the two must never be folded together.
iomgaa