fix: keep a low-tier answer out of the cache slot a max-tier one filled

The per-call reasoning tier never reached the cache key, and the model
fingerprint could not stand in for it: the fingerprint is computed once at
assembly time, so two calls on the same client asking for low and max looked
identical to it. Same messages, different tiers, one shared entry -- the
verbatim replay of issue #4's five seeds all hitting the same response.

Source-level tiers join the fingerprint under the same rule enable_thinking
already follows (appended only when the source takes a position), and the
filter that decides which sources enter the mark set is widened to match --
without that, a source configured with nothing but REASONING_EFFORT would
never reach _fingerprint_mark at all.

None (no opinion) and Effort.NONE (asked not to reason) stay distinct keys.
Sources that opine on neither keep byte-identical keys and fingerprints, so
nothing existing cold-starts.
This commit is contained in:
2026-09-05 02:39:38 -04:00
parent 80a8013642
commit 33c8e8274b
4 changed files with 137 additions and 10 deletions
+15 -1
View File
@@ -17,7 +17,7 @@ from typing import TYPE_CHECKING, Any
from loguru import logger
from polygateway.types import ChatRequest, LLMResponse, ThinkingObservation
from polygateway.types import ChatRequest, Effort, LLMResponse, ThinkingObservation
if TYPE_CHECKING:
from collections.abc import Mapping
@@ -83,12 +83,21 @@ def build_cache_key(
salt: str | None,
*,
sampling: Mapping[str, Any] | None = None,
reasoning_effort: Effort | None = None,
) -> str:
"""缓存 key 公式;salt 仅非 None 时参与(VT 旧键语义: 不传 salt 键形不变)。
`sampling` 仅**非空**时参与(与 salt 的"仅非 None"不同——空串是有意义的
salt,而空采样参数与不传无语义差别)。它必须进 key: 否则同 messages 跑 5 个
seed 会全部命中第一次的响应,标准差恒为 0 且不报错(issue #4 决策 C)。
`reasoning_effort` 是**请求级**档位(issue #20),仅非 `None` 时参与。它不能靠
`model_fingerprint` 代劳: 后者是**装配期**算出的集合级指纹,一次调用改档位不会
让它变一个字节;不进 key 则同 messages 跑 low 与 max 互相命中,是 issue #4
「5 个 seed 全命中同一响应」的逐字翻版。
判据用 `is not None` 而非真值: `Effort.NONE`(明确要求不推理)与 `None`
(不表态)语义不同——前者拿到的是没有推理过程的响应,合并即毒化。
"""
key_obj: dict[str, Any] = {
"model": model_fingerprint,
@@ -99,6 +108,8 @@ def build_cache_key(
key_obj["salt"] = salt
if sampling:
key_obj["sampling"] = dict(sampling)
if reasoning_effort is not None:
key_obj["reasoning_effort"] = str(reasoning_effort)
payload = json.dumps(key_obj, sort_keys=True, ensure_ascii=False)
return _KEY_PREFIX + hashlib.sha256(payload.encode("utf-8")).hexdigest()
@@ -139,6 +150,9 @@ class CacheMW:
namespace,
request.cache_salt,
sampling=request.sampling,
# 请求级档位必须逐次进 key: `self._fingerprint` 是装配期的集合级指纹,
# 同一个 client 上 low 与 max 两次调用在它眼里毫无分别(issue #20)
reasoning_effort=request.reasoning_effort,
)
cached = await self._safe_get(key)
if cached is not None: