Commit Graph

2 Commits

Author SHA1 Message Date
iomgaa 97de4104c6 docs: finalize streaming stall design with independent review errata
B1 chunk = any delta (content or reasoning), B2 telemetry plumbing
via explicit _emit pass, B3 orchestration entry gate for independent
stall hedging; N1-N9 advisory fixes merged. Human decisions recorded
in design section 9.
2026-09-13 23:20:52 -04:00
iomgaa 03cbf4c8db docs: add approved streaming stall measurement and mid-stream hedging design
Findings: industry paradigm research (watchdog-retry is the norm,
mid-stream hedging has no public precedent; pi-tps threshold-based
stall deduction; Gregg off-CPU paired reporting; vLLM exclusion
clauses). Design: generation_ms redefined as net generation time
(wall minus all inter-chunk stalls over threshold), CallStats gains
stall_ms/stall_count, HEDGE__STALL_AFTER_S mid-stream hedge
independent of HEDGE__AFTER_S. Issues #27 #28.
2026-09-13 22:53:00 -04:00