iomgaa
6fafd95d6c
两个自行调研后决定的问题，各自的证据写进了 design doc： 一、handler 返回 str 而不是带截断计数的小结构（0008 决策三，文末新增一节）。三条实据： 两个真实消费者的执行函数今天就返回纯字符串（GovDoc 的 handler 是 Coroutine[..., str]， dissect 的环境 execute 是 -> str）；dissect 的 observation_truncated_chars 唯一的生产写入点 硬编码 0 且全仓零读取点，存在的是名字不是需求；reference/pi 是唯一把截断做完整的，它记的是 totalBytes/outputBytes/maxBytes 这组绝对量而不是一个差值——现在补 truncated_chars 补的 大概率是错形状，正是 scope.md 说的「猜出来的接缝比没有接缝更难拆」。 二、新增 design 0009：src/polyloop/ 下每个数据类都加 kw_only=True，另加一条扫描测试守它。 实验室七个仓库 223 个 dataclass 里 kw_only 出现零次，但那是默认行为不是选择。真正的证据是 PolyGateway：它的 LLMResponse 前 11 个字段顺序被三个下游的测试替身按位置构造锁死，模块 docstring 写着「字段顺序即公共承诺」，还得专门写一条 test_eleven_legacy_fields_positional 守着，从此再也插不进字段。那个约束不是它选的是它继承的，而本库还没有下游装上。 扫描测试查的是构造签名不是那个装饰器参数——要守的承诺是「按位置构造不了」。 executor() 在派生那一刻全查一遍实现，缺一个就报错，不拖到分发时才炸。RegistryExecutor 是 具体类而不是闭包，因为 RunRequest 要用 isinstance 认它。CancelledError 不被那个 except Exception 接住（它继承 BaseException），有测试守着。
602 lines
23 KiB
Python
602 lines
23 KiB
Python
"""工具规格与注册表的行为。
|
||
|
||
这里断言的不是「注册表有哪些方法」，是它对外的行为——名字与签名本身由
|
||
`research-wiki/design/0006-public-names-and-signatures.md` 决策六承诺，改了是破坏性变更。
|
||
|
||
最要紧的一条是**四者同源**：模型看见的 schema、存在性校验、参数校验、分发都由同一个实例
|
||
驱动。收窄之后模型看不见的工具，校验也必须拒绝它——这一条单独写在下面，因为它正是这个
|
||
模块存在的理由。
|
||
"""
|
||
|
||
import json
|
||
from collections.abc import Mapping
|
||
|
||
import pytest
|
||
|
||
from polyloop.ports import Action, ToolCall
|
||
from polyloop.tools import (
|
||
RegistryExecutor,
|
||
ToolEnvironmentError,
|
||
ToolRegistry,
|
||
ToolSpec,
|
||
ToolValidationError,
|
||
)
|
||
from polyloop.types import ActionStatus, ReplayPolicy
|
||
|
||
pytestmark = pytest.mark.unit
|
||
|
||
|
||
def _spec(name: str, **kwargs: object) -> ToolSpec:
|
||
"""造一个规格，只有 `name` 是必须自己填的。"""
|
||
parameters = kwargs.pop("parameters", {})
|
||
return ToolSpec(name=name, description=f"{name} 的说明", parameters=parameters, **kwargs) # type: ignore[arg-type]
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# ToolSpec
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_spec_rejects_an_empty_name() -> None:
|
||
"""空名字在提示词里不可见，模型永远调不到它。"""
|
||
with pytest.raises(ValueError, match="工具名"):
|
||
ToolSpec(name="", description="", parameters={})
|
||
|
||
|
||
def test_spec_rejects_a_blank_name() -> None:
|
||
"""纯空白的名字和空串一样，在提示词里不可见。"""
|
||
with pytest.raises(ValueError, match="工具名"):
|
||
ToolSpec(name=" ", description="", parameters={})
|
||
|
||
|
||
def test_spec_rejects_a_name_that_is_not_a_string() -> None:
|
||
"""名字要落进发给模型的那份 schema，不是字符串的话整个请求会被网关拒掉。
|
||
|
||
那时报错指向请求体、不指向注册表，而两者隔着好几层。
|
||
"""
|
||
with pytest.raises(TypeError, match="工具名"):
|
||
ToolSpec(name=123, description="", parameters={}) # type: ignore[arg-type]
|
||
|
||
|
||
def test_spec_rejects_a_description_that_is_not_a_string() -> None:
|
||
with pytest.raises(TypeError, match="工具说明"):
|
||
ToolSpec(name="search", description=object(), parameters={}) # type: ignore[arg-type]
|
||
|
||
|
||
def test_spec_rejects_non_mapping_parameters() -> None:
|
||
with pytest.raises(TypeError, match="parameters"):
|
||
ToolSpec(name="search", description="", parameters=["query"]) # type: ignore[arg-type]
|
||
|
||
|
||
def test_spec_snapshots_the_parameters_it_was_given() -> None:
|
||
"""注册之后改调用方那份字典，注册表看见的仍是注册那一刻的形状。
|
||
|
||
不拷贝的话，「模型看见的 schema」会随调用方在别处的一次修改一起变，而那次修改没有任何
|
||
地方记录得到——事后翻轨迹，模型当时到底看见的是哪一份，查不出来。
|
||
"""
|
||
schema: dict[str, object] = {"properties": {"query": {"type": "string"}}}
|
||
spec = _spec("search", parameters=schema)
|
||
|
||
schema["properties"] = {"query": {"type": "integer"}} # type: ignore[index]
|
||
|
||
assert spec.parameters["properties"] == {"query": {"type": "string"}}
|
||
|
||
|
||
def test_a_spec_taken_out_of_the_registry_cannot_be_edited_in_place() -> None:
|
||
"""从注册表里取出规格、往里伸一层去改，改不动。
|
||
|
||
只冻最外面一层挡不住这种改法，而它一下同时改掉模型看见的 schema 和校验用的 schema——
|
||
第 1 步模型看到的和第 5 步校验用的就分了岔，而这次修改没有任何地方记录得到。
|
||
"""
|
||
registry = ToolRegistry(
|
||
[
|
||
_spec(
|
||
"read",
|
||
parameters={"properties": {"path": {"type": "string"}}, "required": ["path"]},
|
||
)
|
||
]
|
||
)
|
||
taken = registry.spec_for("read")
|
||
assert taken is not None
|
||
|
||
with pytest.raises(TypeError):
|
||
taken.parameters["required"] = ["path", "mode"] # type: ignore[index]
|
||
with pytest.raises(TypeError):
|
||
taken.parameters["properties"]["path"]["type"] = "integer" # type: ignore[index]
|
||
|
||
registry.validate(ToolCall(name="read", arguments={"path": "a.txt"}))
|
||
|
||
|
||
def test_spec_defaults_are_the_conservative_ones() -> None:
|
||
"""重放策略默认「绝不重放」、完成标记默认「不完成」。
|
||
|
||
两个默认值的方向都是「漏了声明只会多停一次，有人会看见」，反过来则是静默重复副作用、
|
||
或者一次运行永远停不下来。
|
||
"""
|
||
spec = _spec("write_file")
|
||
|
||
assert spec.replay_policy is ReplayPolicy.NEVER
|
||
assert spec.completes_run is False
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 注册与查询
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_registry_rejects_duplicate_names() -> None:
|
||
"""重名会让「模型看见的那一份」和「分发时取到的那一份」取决于注册顺序。"""
|
||
with pytest.raises(ValueError, match="重复"):
|
||
ToolRegistry([_spec("search"), _spec("search")])
|
||
|
||
|
||
def test_registry_rejects_things_that_are_not_specs() -> None:
|
||
with pytest.raises(TypeError):
|
||
ToolRegistry([{"name": "search"}]) # type: ignore[list-item]
|
||
|
||
|
||
def test_an_empty_registry_is_constructible() -> None:
|
||
"""不注册任何工具是一种正常装配：模型输出的是一整段代码，不是工具调用。"""
|
||
registry = ToolRegistry()
|
||
|
||
assert registry.names() == ()
|
||
assert list(registry.schema_for_model()) == []
|
||
|
||
|
||
def test_registration_order_is_preserved() -> None:
|
||
"""工具清单要贴进提示词，顺序必须是确定的。"""
|
||
registry = ToolRegistry([_spec("read"), _spec("write"), _spec("search")])
|
||
|
||
assert registry.names() == ("read", "write", "search")
|
||
assert [entry["name"] for entry in registry.schema_for_model()] == ["read", "write", "search"]
|
||
|
||
|
||
def test_spec_for_returns_none_for_an_unregistered_name() -> None:
|
||
"""问一个不在注册表里的名字是正常情形，不是错误。
|
||
|
||
没有工具的动作（一整段代码）就问不出规格，那时调用方取「绝不重放」。抛异常的话这条
|
||
正常路径要用捕获异常来走。
|
||
"""
|
||
registry = ToolRegistry([_spec("read")])
|
||
|
||
assert registry.spec_for("read") is not None
|
||
assert registry.spec_for("write") is None
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 收窄
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_restrict_to_keeps_registration_order_not_the_caller_order() -> None:
|
||
"""收窄结果按注册顺序排，不按传进来的那个集合的迭代顺序。
|
||
|
||
收窄清单常常是一个 `set`，而 Python 的字符串哈希每进程随机——照集合顺序输出会让同一份
|
||
配置在不同进程里渲染出不同的提示词。
|
||
"""
|
||
registry = ToolRegistry([_spec("read"), _spec("write"), _spec("search")])
|
||
|
||
narrowed = registry.restrict_to({"search", "read"})
|
||
|
||
assert narrowed.names() == ("read", "search")
|
||
|
||
|
||
def test_restrict_to_leaves_the_original_alone() -> None:
|
||
"""注册表是不可变值对象，取子集返回新实例。"""
|
||
registry = ToolRegistry([_spec("read"), _spec("write")])
|
||
|
||
narrowed = registry.restrict_to(["read"])
|
||
|
||
assert narrowed.names() == ("read",)
|
||
assert registry.names() == ("read", "write")
|
||
|
||
|
||
def test_restrict_to_rejects_a_name_it_does_not_have() -> None:
|
||
"""静默丢弃的话，模型看不见调用方以为已经开放的那个工具，而表现是模型在绕圈。"""
|
||
registry = ToolRegistry([_spec("read")])
|
||
|
||
with pytest.raises(ValueError, match="不在注册表里"):
|
||
registry.restrict_to(["read", "reed"])
|
||
|
||
|
||
def test_two_registries_with_the_same_specs_are_equal() -> None:
|
||
"""相等按「注册了哪些规格、什么顺序」判，不按对象身份判。
|
||
|
||
构造运行请求时要比对「执行器持有的注册表」和「本次可见的注册表」，两份内容相同的注册表
|
||
在模型可见 schema 与实际分发上完全一致，没有可失败的地方。
|
||
"""
|
||
specs = [_spec("read"), _spec("write")]
|
||
|
||
assert ToolRegistry(specs) == ToolRegistry(specs)
|
||
assert ToolRegistry(specs) != ToolRegistry(list(reversed(specs)))
|
||
assert ToolRegistry(specs) != ToolRegistry([_spec("read")])
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 模型可见的 schema
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_schema_for_model_is_json_serialisable() -> None:
|
||
"""这份清单要贴进提示词或者发给模型 API，必须能直接序列化。"""
|
||
registry = ToolRegistry(
|
||
[_spec("read", parameters={"properties": {"path": {"type": "string"}}})]
|
||
)
|
||
|
||
json.dumps(list(registry.schema_for_model()))
|
||
|
||
|
||
def test_mutating_the_returned_schema_does_not_touch_the_registry() -> None:
|
||
"""每次调用现造一份新的普通字典，调用方改它不会影响后面几步看见的东西。"""
|
||
registry = ToolRegistry(
|
||
[_spec("read", parameters={"properties": {"path": {"type": "string"}}})]
|
||
)
|
||
|
||
entry = registry.schema_for_model()[0]
|
||
entry["parameters"]["properties"]["path"]["type"] = "integer" # type: ignore[index]
|
||
|
||
assert registry.schema_for_model()[0]["parameters"] == {
|
||
"properties": {"path": {"type": "string"}}
|
||
}
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 校验
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_validate_accepts_a_well_formed_call() -> None:
|
||
registry = ToolRegistry(
|
||
[
|
||
_spec(
|
||
"read",
|
||
parameters={
|
||
"properties": {"path": {"type": "string"}},
|
||
"required": ["path"],
|
||
"additionalProperties": False,
|
||
},
|
||
)
|
||
]
|
||
)
|
||
|
||
registry.validate(ToolCall(name="read", arguments={"path": "a.txt"}))
|
||
|
||
|
||
def test_validate_rejects_a_tool_that_is_not_registered() -> None:
|
||
registry = ToolRegistry([_spec("read")])
|
||
|
||
with pytest.raises(ToolValidationError, match="不存在"):
|
||
registry.validate(ToolCall(name="write", arguments={}))
|
||
|
||
|
||
def test_validate_rejects_a_missing_required_parameter() -> None:
|
||
registry = ToolRegistry(
|
||
[
|
||
_spec(
|
||
"read",
|
||
parameters={"properties": {"path": {"type": "string"}}, "required": ["path"]},
|
||
)
|
||
]
|
||
)
|
||
|
||
with pytest.raises(ToolValidationError, match="必填"):
|
||
registry.validate(ToolCall(name="read", arguments={}))
|
||
|
||
|
||
def test_validate_rejects_an_undeclared_parameter_only_when_the_schema_says_so() -> None:
|
||
"""多出来的键只在 schema 显式写了 `additionalProperties: false` 时才拒绝。
|
||
|
||
JSON Schema 里这一项默认为真，跟着它走；自作主张收紧的话，一份合法 schema 在库里和在别处
|
||
的含义就不一样了，而不一样的地方没有任何标记。
|
||
"""
|
||
lenient = ToolRegistry([_spec("read", parameters={"properties": {"path": {"type": "string"}}})])
|
||
strict = ToolRegistry(
|
||
[
|
||
_spec(
|
||
"read",
|
||
parameters={
|
||
"properties": {"path": {"type": "string"}},
|
||
"additionalProperties": False,
|
||
},
|
||
)
|
||
]
|
||
)
|
||
call = ToolCall(name="read", arguments={"path": "a.txt", "encoding": "utf-8"})
|
||
|
||
lenient.validate(call)
|
||
with pytest.raises(ToolValidationError, match="未声明"):
|
||
strict.validate(call)
|
||
|
||
|
||
def test_validate_rejects_a_wrong_top_level_type() -> None:
|
||
registry = ToolRegistry(
|
||
[_spec("read", parameters={"properties": {"path": {"type": "string"}}})]
|
||
)
|
||
|
||
with pytest.raises(ToolValidationError, match="类型"):
|
||
registry.validate(ToolCall(name="read", arguments={"path": 7}))
|
||
|
||
|
||
def test_a_boolean_is_not_an_integer() -> None:
|
||
"""布尔在 Python 里是整数的子类，在 JSON 里不是。
|
||
|
||
不排掉的话，一个声明收整数的工具会收下 `True`，然后在工具内部被当成 1 用。
|
||
"""
|
||
registry = ToolRegistry([_spec("head", parameters={"properties": {"n": {"type": "integer"}}})])
|
||
|
||
with pytest.raises(ToolValidationError, match="类型"):
|
||
registry.validate(ToolCall(name="head", arguments={"n": True}))
|
||
|
||
|
||
def test_a_float_with_no_fractional_part_counts_as_an_integer() -> None:
|
||
"""JSON Schema draft-06 起，小数部分为零的浮点数是合法的整数。
|
||
|
||
模型写出 `1e2` 或者 `3.0`，`json.loads` 给的就是 float。照「必须是 int」判会拒掉一次
|
||
合法调用，而模型怎么改都过不去——它写的东西按标准就是对的。
|
||
"""
|
||
registry = ToolRegistry([_spec("head", parameters={"properties": {"n": {"type": "integer"}}})])
|
||
|
||
registry.validate(ToolCall(name="head", arguments={"n": 3.0}))
|
||
with pytest.raises(ToolValidationError, match="类型"):
|
||
registry.validate(ToolCall(name="head", arguments={"n": 3.5}))
|
||
|
||
|
||
def test_pattern_properties_switches_off_the_unknown_key_check() -> None:
|
||
"""`patternProperties` 在场时，「哪些键被声明过」要靠正则才答得出，这里不答。
|
||
|
||
照 `properties` 的键去判的话，每一个匹配到 pattern 的键都会被拒——而那些正是这份 schema
|
||
专门要收的键，模型改名字也绕不过去。
|
||
"""
|
||
registry = ToolRegistry(
|
||
[
|
||
_spec(
|
||
"put",
|
||
parameters={
|
||
"patternProperties": {"^x_": {"type": "string"}},
|
||
"additionalProperties": False,
|
||
},
|
||
)
|
||
]
|
||
)
|
||
|
||
registry.validate(ToolCall(name="put", arguments={"x_1": "a"}))
|
||
|
||
|
||
def test_validate_rejects_arguments_that_are_not_a_mapping() -> None:
|
||
"""`ToolCall` 自己不校验字段，一个列表参数能从下游的解释器直接产出。
|
||
|
||
不在这里拦住的话，后面那句 `.items()` 会抛 `AttributeError` 打断整次运行；拦住了它就
|
||
只是一条正常观察，模型收到说明可以自己纠正。
|
||
"""
|
||
registry = ToolRegistry([_spec("read")])
|
||
|
||
with pytest.raises(ToolValidationError, match="映射"):
|
||
registry.validate(ToolCall(name="read", arguments=["a.txt"])) # type: ignore[arg-type]
|
||
|
||
|
||
def test_a_type_may_be_declared_as_a_list_of_alternatives() -> None:
|
||
registry = ToolRegistry(
|
||
[_spec("head", parameters={"properties": {"n": {"type": ["integer", "null"]}}})]
|
||
)
|
||
|
||
registry.validate(ToolCall(name="head", arguments={"n": 3}))
|
||
registry.validate(ToolCall(name="head", arguments={"n": None}))
|
||
with pytest.raises(ToolValidationError, match="类型"):
|
||
registry.validate(ToolCall(name="head", arguments={"n": "3"}))
|
||
|
||
|
||
def test_an_empty_schema_accepts_anything() -> None:
|
||
"""没声明参数就等于不约束参数，不等于「不许带参数」。"""
|
||
registry = ToolRegistry([_spec("ping")])
|
||
|
||
registry.validate(ToolCall(name="ping", arguments={"anything": [1, 2, 3]}))
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 四者同源
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
def test_narrowing_hides_a_tool_from_the_model_and_from_validation_together() -> None:
|
||
"""收窄之后模型看不见的工具，校验也拒绝它。
|
||
|
||
这是这个模块存在的理由：模型看见的 schema、存在性校验、参数校验、分发四者同源。不同源
|
||
的表现是「模型调了一个它看得见的工具却说不存在」，或者反过来——校验放行了一个分发时
|
||
找不到的名字。
|
||
"""
|
||
registry = ToolRegistry([_spec("read"), _spec("write")])
|
||
|
||
narrowed = registry.restrict_to(["read"])
|
||
|
||
assert [entry["name"] for entry in narrowed.schema_for_model()] == ["read"]
|
||
assert narrowed.spec_for("write") is None
|
||
with pytest.raises(ToolValidationError):
|
||
narrowed.validate(ToolCall(name="write", arguments={}))
|
||
|
||
|
||
# ---------------------------------------------------------------------------
|
||
# 派生分发器
|
||
# ---------------------------------------------------------------------------
|
||
|
||
|
||
async def _echo(arguments: Mapping[str, object]) -> str:
|
||
return f"echo {arguments}"
|
||
|
||
|
||
def _tool_action(name: str, **arguments: object) -> Action:
|
||
return Action(text=f"{name}(...)", tool_call=ToolCall(name=name, arguments=arguments))
|
||
|
||
|
||
def test_deriving_an_executor_needs_every_tool_to_have_an_implementation() -> None:
|
||
"""缺实现在派生那一刻就报错，不等到分发时才发现。
|
||
|
||
分发时才发现的话，那是运行到第几步才炸，而前几步已经花了钱、留了轨迹，而且不同的运行会
|
||
在不同的步数上炸。
|
||
"""
|
||
registry = ToolRegistry([_spec("read", handler=_echo), _spec("write")])
|
||
|
||
with pytest.raises(ValueError, match="没有实现"):
|
||
registry.executor()
|
||
|
||
|
||
def test_a_registry_without_handlers_is_still_a_valid_registry() -> None:
|
||
"""项目自己写动作执行器时，工具清单照样进模型可见的 schema、照样被校验。
|
||
|
||
实现不在库这边，那时也不该调 `executor()`。必填实现的话，这种项目要给每个工具写一个
|
||
永远不会被调用的空壳。
|
||
"""
|
||
registry = ToolRegistry([_spec("read")])
|
||
|
||
assert registry.names() == ("read",)
|
||
registry.validate(ToolCall(name="read", arguments={}))
|
||
|
||
|
||
def test_an_executor_holds_the_registry_it_came_from() -> None:
|
||
"""运行请求靠 `isinstance` 认出它、再比对注册表，所以那份注册表必须拿得到。"""
|
||
registry = ToolRegistry([_spec("read", handler=_echo)])
|
||
|
||
executor = registry.executor()
|
||
|
||
assert isinstance(executor, RegistryExecutor)
|
||
assert executor.registry == registry
|
||
|
||
|
||
def test_the_executor_reports_the_tool_names_as_its_parameters() -> None:
|
||
"""工具集是「模型看得见的东西」的一部分，要能进参数快照。
|
||
|
||
换一组工具续跑而快照不比对，前几步与后几步的可选动作集就不一样了，而两段轨迹在文件里
|
||
看起来是同一次运行。
|
||
"""
|
||
registry = ToolRegistry([_spec("read", handler=_echo), _spec("write", handler=_echo)])
|
||
|
||
assert registry.executor().parameters() == {"tools": "read,write"}
|
||
|
||
|
||
async def test_a_successful_call_is_executed_with_the_handler_text() -> None:
|
||
registry = ToolRegistry([_spec("read", handler=_echo)])
|
||
|
||
outcome = await registry.executor().execute(_tool_action("read", path="a.txt"))
|
||
|
||
assert outcome.status is ActionStatus.EXECUTED
|
||
assert outcome.observation == "echo {'path': 'a.txt'}"
|
||
assert outcome.observation_is_synthetic is False
|
||
assert outcome.observation_truncated_chars == 0
|
||
|
||
|
||
async def test_the_env_completion_signal_is_always_false_on_this_path() -> None:
|
||
"""这条路径上根本没有环境可问，注册表派生的执行器手上只有一份工具清单。
|
||
|
||
走这条路的运行靠完成标记收尾，而那一档由停止判定去查注册表。填成真会凭空造出一条环境侧
|
||
证据，而两条完成通路的可信度本来就不同。
|
||
"""
|
||
registry = ToolRegistry([_spec("submit", handler=_echo, completes_run=True)])
|
||
|
||
outcome = await registry.executor().execute(_tool_action("submit"))
|
||
|
||
assert outcome.env_reported_completion is False
|
||
|
||
|
||
async def test_an_action_without_a_tool_call_is_not_executed() -> None:
|
||
"""一个只认工具调用的执行器收到一段代码，说明这次运行把两种动作语言配串了。
|
||
|
||
判成未执行而不是抛异常：抛异常会终止整次运行，而这一档留一条记录，事后能看见它发生
|
||
过几次。
|
||
"""
|
||
registry = ToolRegistry([_spec("read", handler=_echo)])
|
||
|
||
outcome = await registry.executor().execute(Action(text="print(1)", tool_call=None))
|
||
|
||
assert outcome.status is ActionStatus.NOT_EXECUTED
|
||
|
||
|
||
async def test_an_invalid_call_is_not_executed_and_never_reaches_the_handler() -> None:
|
||
"""工具不存在或参数不合法时直接合成「未执行」，不经过任何实现。"""
|
||
called = False
|
||
|
||
async def _never(arguments: Mapping[str, object]) -> str:
|
||
nonlocal called
|
||
called = True
|
||
return ""
|
||
|
||
registry = ToolRegistry([_spec("read", handler=_never)])
|
||
|
||
outcome = await registry.executor().execute(_tool_action("reed"))
|
||
|
||
assert outcome.status is ActionStatus.NOT_EXECUTED
|
||
assert called is False
|
||
|
||
|
||
async def test_a_plain_exception_from_a_tool_is_a_normal_observation() -> None:
|
||
"""工具里抛一个普通异常算「已执行」，原样回喂让模型自己纠正。
|
||
|
||
判成「工具无效、不计有效动作」的话，模型能无限重试同一个坏工具直到把步数上限耗尽。
|
||
观察带类名与文本、不带调用栈——模型要的是「哪里错了」，调用栈对它没用，还会把库内部的
|
||
路径喂进提示词。
|
||
"""
|
||
|
||
async def _boom(arguments: Mapping[str, object]) -> str:
|
||
raise ValueError("参数解析失败")
|
||
|
||
registry = ToolRegistry([_spec("read", handler=_boom)])
|
||
|
||
outcome = await registry.executor().execute(_tool_action("read"))
|
||
|
||
assert outcome.status is ActionStatus.EXECUTED
|
||
assert outcome.observation == "ValueError: 参数解析失败"
|
||
|
||
|
||
async def test_only_a_tool_environment_error_means_the_environment_broke() -> None:
|
||
"""分界线是环境还能不能接着服务，不是「有没有抛异常」。
|
||
|
||
没有这个口子的话，派生分发器永远产不出环境故障，于是后端挂掉时模型会一遍遍重试、把预算
|
||
烧光，而轨迹上表现成「预算耗尽」。
|
||
"""
|
||
|
||
async def _down(arguments: Mapping[str, object]) -> str:
|
||
raise ToolEnvironmentError("后端连不上")
|
||
|
||
registry = ToolRegistry([_spec("read", handler=_down)])
|
||
|
||
outcome = await registry.executor().execute(_tool_action("read"))
|
||
|
||
assert outcome.status is ActionStatus.ENV_ERROR
|
||
assert outcome.observation == "后端连不上"
|
||
|
||
|
||
async def test_cancellation_passes_straight_through() -> None:
|
||
"""`CancelledError` 不被那个 `except Exception` 接住——它继承的是 `BaseException`。
|
||
|
||
吞掉它的后果不是「取消失败」这么直白，是容器租约、连接和临时目录持续泄漏，而且一声
|
||
不吭。
|
||
"""
|
||
import asyncio
|
||
|
||
async def _cancelled(arguments: Mapping[str, object]) -> str:
|
||
raise asyncio.CancelledError
|
||
|
||
registry = ToolRegistry([_spec("read", handler=_cancelled)])
|
||
|
||
with pytest.raises(asyncio.CancelledError):
|
||
await registry.executor().execute(_tool_action("read"))
|
||
|
||
|
||
async def test_a_handler_that_returns_the_wrong_type_fails_loudly() -> None:
|
||
"""返回类型不对是实现的签名写错了，不是模型能应对的运行时状况。
|
||
|
||
它在第一次调用就必然发生，也就是在写这个工具的人第一次跑测试时，不会拖到生产的第 40 步。
|
||
放过去的话，一个非字符串会进到持久化的观察字段里。
|
||
"""
|
||
|
||
async def _wrong(arguments: Mapping[str, object]) -> str:
|
||
return {"not": "a string"} # type: ignore[return-value]
|
||
|
||
registry = ToolRegistry([_spec("read", handler=_wrong)])
|
||
|
||
with pytest.raises(TypeError, match="必须返回一段字符串观察"):
|
||
await registry.executor().execute(_tool_action("read"))
|
||
|
||
|
||
def test_a_handler_must_be_callable() -> None:
|
||
with pytest.raises(TypeError, match="handler"):
|
||
_spec("read", handler="not callable")
|