feat: refuse an impossible tier with the cheapest one that model does have
resolve_thinking now takes an Effort instead of a tri-state bool, and the four gates become five. The new one sits ahead of the generic tier check on purpose: asking for `none` on GLM-5.3 used to fall through to "none is not supported, pick low/high/max", which loses both the fact that the model cannot stop reasoning and the one tier the caller could switch to right now. Without that alternative, downstream goes looking for extra_body — which is how issue #20 happened in the first place. The return type is a ThinkingResolution rather than the payload alone. Under fallback="nearest" the tier that goes out is not the tier that was asked for, and telemetry has to record the one that ran, or task 10 files a call under a tier it never used. Ties in that mapping go to the weaker side: a silent medium -> max is a multiple of the bill, and the library does not raise a caller's price on its own. Two readings the design left implicit, both settled the way its own compatibility promise requires: - `auto` is exempt from the tier list. It means "on, no tier named", which in the body is the absence of the effort key, not a value of it. Checking it against the list would break every existing source that sets ENABLE_THINKING=true against deepseek-v4 or glm-5.3. - `none` is never a mapping target. Turning "think less" into "do not think" reverses the decision instead of cheapening it; a switch-only model maps to `auto` and a model that only has `none` still errors. Both call sites convert enable_thinking in place for now; task 5 folds that into effective_effort along with the source- and call-level tiers.
This commit is contained in:
@@ -31,6 +31,7 @@ from polygateway.providers import (
|
||||
from polygateway.telemetry.schema import telemetry_schema_sql
|
||||
from polygateway.thinking import (
|
||||
ThinkingCapability,
|
||||
ThinkingResolution,
|
||||
ThinkingUnsupportedError,
|
||||
get_capability,
|
||||
register_capability,
|
||||
@@ -82,6 +83,7 @@ __all__ = [
|
||||
"TelemetryStatus",
|
||||
"ThinkingCapability",
|
||||
"ThinkingObservation",
|
||||
"ThinkingResolution",
|
||||
"ThinkingUnsupportedError",
|
||||
"ThinkingWire",
|
||||
"TransientError",
|
||||
|
||||
@@ -38,6 +38,7 @@ from polygateway.thinking import get_capability, resolve_thinking
|
||||
from polygateway.transports.openai_compat import OpenAICompatTransport
|
||||
from polygateway.types import (
|
||||
ChatRequest,
|
||||
Effort,
|
||||
LLMResponse,
|
||||
TelemetryStatus,
|
||||
validate_caller_dimensions,
|
||||
@@ -80,10 +81,15 @@ def _guard_thinking(
|
||||
就带着指路信息炸掉。`get_provider` 现在就是同一形态的双点调用。
|
||||
"""
|
||||
for source, profile in zip(sources, profiles, strict=True):
|
||||
# `enable_thinking` 的档位语法糖(True → auto,False → none,None 不表态);
|
||||
# 两个调用点各自就地转换是过渡形态,T5 起由 thinking.effective_effort()
|
||||
# 统一收口并接上源级/请求级档位(设计 §4.2)
|
||||
enabled = source.enable_thinking
|
||||
effort = None if enabled is None else (Effort.AUTO if enabled else Effort.NONE)
|
||||
resolve_thinking(
|
||||
profile,
|
||||
get_capability(source.model, table=capabilities),
|
||||
source.enable_thinking,
|
||||
effort,
|
||||
model=source.model,
|
||||
)
|
||||
|
||||
|
||||
+203
-31
@@ -291,62 +291,234 @@ def register_capability(
|
||||
return table
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ThinkingResolution:
|
||||
"""请求体注入片段 + 本次**实际**生效的档位(设计 §4.1)。
|
||||
|
||||
返回 dataclass 而非裸 Mapping,是因为 `nearest` 映射后"请求的档"与"真正发出
|
||||
去的档"会分叉(请求 `medium`、模型只有 low/high/max → 实际发 `low`)。遥测必
|
||||
须记后者: 记请求档会让按档位分组的压测把整行数据挂在一个从未真正发出过的档
|
||||
下,而那种数据错得看不出来。
|
||||
|
||||
`applied_effort is None` 只出现在 Phase 1(调用方不表态): 库既不注入,也不
|
||||
去推定模型自己的默认档——"没看见"不许说成"发生了"。
|
||||
"""
|
||||
|
||||
payload: Mapping[str, Any]
|
||||
applied_effort: Effort | None
|
||||
|
||||
|
||||
def resolve_thinking(
|
||||
profile: ProviderProfile,
|
||||
capability: ThinkingCapability | None,
|
||||
enable_thinking: bool | None,
|
||||
effort: Effort | None,
|
||||
*,
|
||||
model: str,
|
||||
fallback: str = "error",
|
||||
warn_unregistered: bool = True,
|
||||
) -> Mapping[str, Any]:
|
||||
"""三态 + 两层能力 → 请求体注入片段;不可满足时 ValueError。
|
||||
) -> ThinkingResolution:
|
||||
"""档位 + 两层声明(形态/能力)→ 注入片段;不可满足时 `ThinkingUnsupportedError`。
|
||||
|
||||
调用点负责翻译: 装配期直接冒泡(配置错误),transport 内翻译为
|
||||
`RequestRejectedError`(四分类之一)。判定顺序即语义,不可调换——形态未知时
|
||||
无从注入,能力如何无关紧要,故 Phase 2 必须先于 Phase 4;未登记模型没有
|
||||
`can_disable` 可读,故 Phase 3 必须先于 Phase 4。
|
||||
`RequestRejectedError`(四分类之一)。**判定顺序即语义,不可调换**:
|
||||
|
||||
========== ================================================================
|
||||
Phase 1 不表态 → 不注入。与 `Effort.NONE` 严格区分: 前者是"随模型默认",
|
||||
后者是"要求不推理"
|
||||
Phase 2 形态未知 → 报错。无从注入时,模型能力如何都无关紧要,故必须先于
|
||||
4/5
|
||||
Phase 3 能力未登记 → 尽力注入且**不校验档位**。没有清单可比对,拿空清单
|
||||
去拒绝档位就是凭空报错;新模型上线不该被库挡住(设计 §5 R4)
|
||||
Phase 4 请求 `none` 而模型关不掉 → 报错并给出 `cheapest_effort`
|
||||
Phase 5 其余档位打空 → 报错(或按 `fallback` 映射)
|
||||
========== ================================================================
|
||||
|
||||
**4 必须先于 5**: `none` 只是 5 的一个特例,若让它落进 5 的通用分支,报错就
|
||||
退化成"不支持 none,可选 low/high/max"——丢掉"这个模型根本关不掉"这个关键
|
||||
信息与可执行替代,下游随后就会去找 `extra_body` 那条绕过的路,而那正是
|
||||
issue #20 的成因。
|
||||
|
||||
**`auto` 不受档位清单约束**: 它表达的是"开启,但不指定强度",在请求体里就是
|
||||
"不写 `effort_key`",而不是写进 `effort_key` 的某个取值,故 Phase 5 放行它。
|
||||
反过来判会让存量的 `ENABLE_THINKING=true`(T5 起等价于 `auto`)在 deepseek-v4
|
||||
与 glm-5.3 这类清单里没有 `auto` 的模型上当场报错,而设计 §12 明确承诺存量
|
||||
配置继续可跑——那里唯一允许新报错的是"关闭一个官方不可关的模型"。
|
||||
|
||||
`model` 只用于错误与告警文案: 报错能定位到具体模型才有可操作性,而
|
||||
`capability` 为 None(未登记)时无从从别处取得模型名。
|
||||
|
||||
`warn_unregistered=False` 供请求热路径去重用: 装配期已经喊过一次,逐次
|
||||
调用再喊只会刷屏。判定结果不受此参数影响。
|
||||
`fallback="nearest"` 是 Phase 5 的逃生口,**默认关闭的理由是钱**: 一次静默的
|
||||
`medium → max` 在 GLM-5.3 上是数倍账单(P5"严禁默认值掩盖错误")。
|
||||
|
||||
`warn_unregistered=False` 供请求热路径去重用: 装配期已经喊过一次,逐次调用
|
||||
再喊只会刷屏。判定结果不受此参数影响。
|
||||
"""
|
||||
# Phase 1: 调用方不表态 —— 与 False 严格区分,用模型默认档
|
||||
if enable_thinking is None:
|
||||
return {}
|
||||
# Phase 1: 调用方不表态 —— 与 Effort.NONE 严格区分,用模型自己的默认档
|
||||
if effort is None:
|
||||
return ThinkingResolution({}, None)
|
||||
wire = profile.thinking
|
||||
slot = wire.on_base if enable_thinking else wire.off
|
||||
direction = "on_base" if enable_thinking else "off"
|
||||
# Phase 2: 形态未知 —— 提供了开关却不知道怎么发,静默放行就是欺骗调用方
|
||||
if slot is None:
|
||||
# Phase 2: 形态未知 —— 给了档位却不知道怎么发,静默放行就是欺骗调用方
|
||||
if wire.on_base is None:
|
||||
raise ThinkingUnsupportedError(
|
||||
f"provider {profile.name!r} 的 {direction} 形态未知(模型 {model!r}): "
|
||||
f"本库不知道该 provider 如何表达这一档。请用 register_provider 注册形态,"
|
||||
f"或改用 SourceConfig.extra_body 直接下发供应商参数"
|
||||
f"provider {profile.name!r} 的推理形态未知(模型 {model!r},请求档位 "
|
||||
f"{effort.value!r}): 本库不知道该 provider 如何表达推理。请用 "
|
||||
f"register_provider 注册形态,或改用 SourceConfig.extra_body 直接下发供应商参数"
|
||||
)
|
||||
# Phase 3: 能力未登记 —— 新模型上线不该被库挡住,但也不该假装成功
|
||||
if capability is None:
|
||||
payload = _inject(profile, wire.on_base, effort, model=model)
|
||||
if warn_unregistered:
|
||||
_warn_unregistered(model, profile, slot)
|
||||
return slot
|
||||
# Phase 4: 明确不支持关闭 —— 调用方要的是"不推理"的语义保证,给不了必须说
|
||||
if enable_thinking is False and not capability.can_disable:
|
||||
_warn_unregistered(model, profile, effort, payload)
|
||||
return ThinkingResolution(payload, effort)
|
||||
# Phase 4: 明确关不掉 —— 调用方要的是"不推理"的语义保证,给不了必须说,且必须
|
||||
# 带一条能立刻照做的替代(见 docstring: 4 先于 5 的理由)
|
||||
if effort is Effort.NONE and not capability.can_disable:
|
||||
raise ThinkingUnsupportedError(_cannot_disable(model, capability))
|
||||
# Phase 5: 档位打空 —— 报错或按 fallback 映射(auto 例外,见 docstring)
|
||||
applied = _settle_tier(effort, capability, model=model, fallback=fallback)
|
||||
return ThinkingResolution(_inject(profile, wire.on_base, applied, model=model), applied)
|
||||
|
||||
|
||||
def _settle_tier(
|
||||
effort: Effort, capability: ThinkingCapability, *, model: str, fallback: str
|
||||
) -> Effort:
|
||||
"""Phase 5: 请求档在不在清单里;不在则按 `fallback` 映射或报错,返回**实际**档。
|
||||
|
||||
`auto` 直接放行: 它不是写进 `effort_key` 的取值,而是"不写 effort_key"
|
||||
(理由见 `resolve_thinking` 的 docstring)。
|
||||
"""
|
||||
if effort is Effort.AUTO or effort in capability.supported_efforts:
|
||||
return effort
|
||||
mapped = _nearest_effort(effort, capability) if fallback == "nearest" else None
|
||||
if mapped is None:
|
||||
raise ThinkingUnsupportedError(
|
||||
f"模型 {model!r} 无法关闭推理,enable_thinking=False 无法满足: "
|
||||
f"{capability.evidence}。该模型的推理是固有属性,任何参数都关不掉——"
|
||||
f"需要关闭思维链请换用支持关闭的模型"
|
||||
_tier_unsupported(model, effort, capability, fallback=fallback)
|
||||
)
|
||||
return slot
|
||||
|
||||
|
||||
def _warn_unregistered(model: str, profile: ProviderProfile, slot: Mapping[str, Any]) -> None:
|
||||
logger.warning(
|
||||
"模型 {} 的推理能力未登记,按 provider {} 的形态尽力注入 {};"
|
||||
"模型 {} 不支持 reasoning_effort={},按 effort_fallback=nearest 改用最近的 {} 档;"
|
||||
"本次真正发出去的是后者,遥测与缓存 key 记的也是后者",
|
||||
model,
|
||||
effort.value,
|
||||
mapped.value,
|
||||
)
|
||||
return mapped
|
||||
|
||||
|
||||
def _inject(
|
||||
profile: ProviderProfile, on_base: Mapping[str, Any], effort: Effort, *, model: str
|
||||
) -> Mapping[str, Any]:
|
||||
"""按 wire 把档位写成请求体片段;wire 表达不了这一档时报错。
|
||||
|
||||
`on_base` 由调用方传入而非在此重读,是为了让"Phase 2 已保证它不是 None"这条
|
||||
前提在签名上看得见。
|
||||
|
||||
三种 `None` 的语义在此**各自兑现**(ThinkingWire 的 docstring 定义了它们):
|
||||
`off is None` = 该 provider 关不掉,`effort_key is None` = 它只有开关没有档位。
|
||||
两者都不是"形态未知",故都不指向 `register_provider`——指错了排查方向比不指
|
||||
还糟。
|
||||
"""
|
||||
wire = profile.thinking
|
||||
if effort is Effort.NONE:
|
||||
if wire.off is None:
|
||||
raise ThinkingUnsupportedError(
|
||||
f"provider {profile.name!r} 没有关闭形态(模型 {model!r}): "
|
||||
f"本库知道它如何表达开启,但该 provider 没有可用的关闭参数。"
|
||||
f"需要不推理请换用支持关闭的 provider 或模型"
|
||||
)
|
||||
return wire.off
|
||||
if effort is Effort.AUTO:
|
||||
# auto = 开启但不指定强度: 逐字节等于升级前的 `thinking_on`
|
||||
return on_base
|
||||
if wire.effort_key is None:
|
||||
raise ThinkingUnsupportedError(
|
||||
f"provider {profile.name!r} 只有推理开关、没有档位键(模型 {model!r}),"
|
||||
f"表达不了 reasoning_effort={effort.value!r}: 请改用 auto/none 两档,"
|
||||
f"或用 register_provider 给该 provider 注册 effort_key"
|
||||
)
|
||||
return {**on_base, wire.effort_key: effort.value}
|
||||
|
||||
|
||||
def _cannot_disable(model: str, capability: ThinkingCapability) -> str:
|
||||
"""Phase 4 的文案: 报错必须带一条能立刻照做的替代,否则等于把用户推回起点。
|
||||
|
||||
只报"关不掉"而不给出路,下游就会去找 `extra_body` 那条绕过库的路——issue #20
|
||||
的成因正是如此。故文案必须含 `cheapest_effort` 的值与 env 键名两样东西。
|
||||
"""
|
||||
# Phase 4 只在 none 不在清单里时触发,而清单构造期保证非空,故必有一档可推荐
|
||||
alternative = capability.cheapest_effort
|
||||
assert alternative is not None
|
||||
return (
|
||||
f"模型 {model!r} 无法关闭推理,reasoning_effort='none' 无法满足: "
|
||||
f"{capability.evidence}。最省的开启档是 {alternative.value!r}——请配 "
|
||||
f"{{SCOPE}}__{{PROVIDER}}__{{N}}__REASONING_EFFORT={alternative.value},"
|
||||
f"或调用时传 reasoning_effort=Effort.{alternative.name};"
|
||||
f"真正需要不推理请换用支持关闭的模型"
|
||||
)
|
||||
|
||||
|
||||
def _tier_unsupported(
|
||||
model: str, effort: Effort, capability: ThinkingCapability, *, fallback: str
|
||||
) -> str:
|
||||
"""Phase 5 的文案: 按 `is_tiered` 分叉,纯开关型模型不能被告知"可选档位"。
|
||||
|
||||
它没有档位——对它说"可选档位: none, auto"是把开关说成了强度轴,下游照着找
|
||||
档位只会一无所获(设计 §3.2 第三个派生量的用途就是这一句话该怎么说)。
|
||||
"""
|
||||
listed = ", ".join(e.value for e in _ordered(capability.supported_efforts))
|
||||
head = f"模型 {model!r} 不支持 reasoning_effort={effort.value!r}: {capability.evidence}。"
|
||||
body = (
|
||||
f"该模型的可选档位: {listed}"
|
||||
if capability.is_tiered
|
||||
else f"该模型只有开关、没有强度档位,可用: {listed}"
|
||||
)
|
||||
# 已经开着 nearest 还走到这里,说明映射本身无解,再劝一遍是废话
|
||||
hint = "" if fallback == "nearest" else ";若希望自动落到最近的档,请配 EFFORT_FALLBACK=nearest"
|
||||
return f"{head}{body}{hint}"
|
||||
|
||||
|
||||
def _ordered(efforts: tuple[Effort, ...]) -> list[Effort]:
|
||||
"""按由弱到强列出档位;`auto` 不在强弱轴上,排在末尾。"""
|
||||
ordered = [e for e in EFFORT_ORDER if e in efforts]
|
||||
if Effort.AUTO in efforts:
|
||||
ordered.append(Effort.AUTO)
|
||||
return ordered
|
||||
|
||||
|
||||
def _nearest_effort(requested: Effort, capability: ThinkingCapability) -> Effort | None:
|
||||
"""取距 `requested` 位序最近的**开启档**;等距取弱侧,无开启档时返回 None。
|
||||
|
||||
候选**剔除 `none`**: 把"想得浅一点"映射成"别想了"是方向反转而非省钱,正是
|
||||
issue #20 那种静默失效的翻版。`none` 的领域归 Phase 4,它在那里已经被处理过,
|
||||
走不到这里(能关就不会打空,不能关就已经报错)。
|
||||
|
||||
`auto` 不在强弱轴上(`EFFORT_ORDER` 不含它),故不参与距离计算,只在一个强度
|
||||
档都没有时兜底——它恰好是纯开关型模型唯一能表达"开"的档。
|
||||
|
||||
**等距取弱**的理由是钱: 一次静默的 `medium → max` 在 GLM-5.3 上是数倍账单,
|
||||
库不替下游涨价。
|
||||
"""
|
||||
candidates = [
|
||||
e for e in EFFORT_ORDER if e is not Effort.NONE and e in capability.supported_efforts
|
||||
]
|
||||
if not candidates:
|
||||
return Effort.AUTO if Effort.AUTO in capability.supported_efforts else None
|
||||
target = EFFORT_ORDER.index(requested)
|
||||
# 排序键第二位是位序本身: 距离相同时位序小的(更省的)胜出
|
||||
return min(
|
||||
candidates, key=lambda e: (abs(EFFORT_ORDER.index(e) - target), EFFORT_ORDER.index(e))
|
||||
)
|
||||
|
||||
|
||||
def _warn_unregistered(
|
||||
model: str, profile: ProviderProfile, effort: Effort, payload: Mapping[str, Any]
|
||||
) -> None:
|
||||
logger.warning(
|
||||
"模型 {} 的推理能力未登记,按 provider {} 的形态尽力注入 {}(请求档位 {});"
|
||||
"若该模型实际不支持这一档,本次设置将静默失效。实测后请用 register_capability 登记",
|
||||
model,
|
||||
profile.name,
|
||||
dict(slot),
|
||||
dict(payload),
|
||||
effort.value,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -34,7 +34,12 @@ from polygateway.thinking import (
|
||||
resolve_thinking,
|
||||
)
|
||||
from polygateway.transports._http_errors import compose_message, summarize_body
|
||||
from polygateway.types import EmbeddingTransportResult, SourceConfig, TransportResult
|
||||
from polygateway.types import (
|
||||
Effort,
|
||||
EmbeddingTransportResult,
|
||||
SourceConfig,
|
||||
TransportResult,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import AsyncIterator, Callable, Mapping
|
||||
@@ -355,14 +360,19 @@ class OpenAICompatTransport:
|
||||
capability = get_capability(source.model, table=self._capabilities)
|
||||
first_time = source.model not in self._warned_models
|
||||
self._warned_models.add(source.model)
|
||||
# `enable_thinking` 的档位语法糖(True → auto,False → none,None 不表态);
|
||||
# 就地转换是过渡形态,T5 起由 thinking.effective_effort() 统一收口并接上
|
||||
# 源级/请求级档位(设计 §4.2)
|
||||
enabled = source.enable_thinking
|
||||
effort = None if enabled is None else (Effort.AUTO if enabled else Effort.NONE)
|
||||
payload.update(
|
||||
resolve_thinking(
|
||||
profile,
|
||||
capability,
|
||||
source.enable_thinking,
|
||||
effort,
|
||||
model=source.model,
|
||||
warn_unregistered=first_time,
|
||||
)
|
||||
).payload
|
||||
)
|
||||
# 顺序即优先级(issue #4 设计决策 A): 配置级 extra_body 在前,调用级
|
||||
# overlay(含结构化注入)在后覆盖之。两行不可调换
|
||||
|
||||
Reference in New Issue
Block a user