feat: let one call ask for a different tier than its source defaults to
The three-layer priority (call > source > enable_thinking sugar > silence) now lives in one pure function, thinking.effective_effort(). The assembly guard and the request hot path used to each carry their own inline copy of the sugar conversion; two copies of the same judgement drift into the worst shape there is - passes at assembly, raises at runtime. The guard now also honours effort_fallback, so a source that opted into nearest is no longer sentenced at assembly for a tier it could have mapped.
This commit is contained in:
@@ -34,7 +34,7 @@ from polygateway.sources import (
|
||||
RoundRobinSelector,
|
||||
SourceCooldownMemo,
|
||||
)
|
||||
from polygateway.thinking import get_capability, resolve_thinking
|
||||
from polygateway.thinking import effective_effort, get_capability, resolve_thinking
|
||||
from polygateway.transports.openai_compat import OpenAICompatTransport
|
||||
from polygateway.types import (
|
||||
ChatRequest,
|
||||
@@ -81,16 +81,20 @@ def _guard_thinking(
|
||||
就带着指路信息炸掉。`get_provider` 现在就是同一形态的双点调用。
|
||||
"""
|
||||
for source, profile in zip(sources, profiles, strict=True):
|
||||
# `enable_thinking` 的档位语法糖(True → auto,False → none,None 不表态);
|
||||
# 两个调用点各自就地转换是过渡形态,T5 起由 thinking.effective_effort()
|
||||
# 统一收口并接上源级/请求级档位(设计 §4.2)
|
||||
enabled = source.enable_thinking
|
||||
effort = None if enabled is None else (Effort.AUTO if enabled else Effort.NONE)
|
||||
resolve_thinking(
|
||||
profile,
|
||||
get_capability(source.model, table=capabilities),
|
||||
effort,
|
||||
# 装配期看不见请求级档位(它逐次调用才产生),故只解源级两层;请求级
|
||||
# 只能在运行期由 transport 校验(设计 §10 的装配期/运行期分工)
|
||||
effective_effort(
|
||||
request_effort=None,
|
||||
source_effort=source.reasoning_effort,
|
||||
enable_thinking=source.enable_thinking,
|
||||
),
|
||||
model=source.model,
|
||||
# 与 transport 用同一个 fallback,否则配了 nearest 的源会在装配期就被
|
||||
# 判死,而它在运行期本来是能映射到最近档跑起来的
|
||||
fallback=source.effort_fallback,
|
||||
)
|
||||
|
||||
|
||||
@@ -295,6 +299,7 @@ class GatewayClient:
|
||||
structured: type[BaseModel] | Literal["json"] | None = None,
|
||||
stream: bool = True,
|
||||
overlay: Mapping[str, Any] | None = None,
|
||||
reasoning_effort: Effort | None = None,
|
||||
tenant_id: str | None = None,
|
||||
meta: Mapping[str, Any] | None = None,
|
||||
) -> LLMResponse:
|
||||
@@ -304,6 +309,10 @@ class GatewayClient:
|
||||
高于源级 `extra_body`、低于结构化输出的注入。带默认值的 keyword-only
|
||||
参数不影响既有调用点(issue #4)。
|
||||
|
||||
`reasoning_effort` 是本次调用的推理档位,优先级高于源级 `REASONING_EFFORT`
|
||||
与 `ENABLE_THINKING`(设计 §4.2)。`None` 是**不表态**(随源级配置),与
|
||||
`Effort.NONE`("要求不推理")严格区分。
|
||||
|
||||
`tenant_id` 与 `meta` 是调用方自定义维度,只进遥测、**不进缓存 key**
|
||||
(租户隔离由 `cache_namespace` 负责,ARCH §7.5);前者享有真实列待遇
|
||||
(可挂 RLS、可进复合索引),后者是任意 KV 容器(issue #11)。
|
||||
@@ -333,6 +342,7 @@ class GatewayClient:
|
||||
stream=stream,
|
||||
overlay=sampling,
|
||||
sampling=sampling,
|
||||
reasoning_effort=reasoning_effort,
|
||||
tenant_id=dimension_tenant_id,
|
||||
meta=dimensions,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user