Files
PolyGateway/research-wiki/designs
iomgaa ac7c86fdee docs: fold independent review into est_tokens design
The reviewer found two real defects. First, changing the usage fallback
to (0, 0) breaks the success-side settlement too, not just the failure
side: retry.py:338 and embedding.py:271 take actual from the same
return value, so a call whose gateway never sends a usage frame would
have its whole pre-deduction refunded -- systematic TPM undercounting.
Added as change item 9. Second, dropping the OCR item: types.py:51 and
ocr.py:9 both state OCR's zero token count is a fact, not an unknown,
so "measured" was already accurate, and relabelling it would pollute
the very metric used to justify the chosen option.

Also pins the cost short-circuit after the cache_hit branch, confines
value-domain enforcement to producers so no bare ValueError escapes
chat(), completes the authoritative-document list, and narrows the
p90 rejection to the read-port argument.
2026-07-30 05:06:00 -04:00
..
2026-07-21 21:39:49 -04:00