docs: add high-correctness warning to maintenance pool design

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-14 10:13:57 -04:00
parent 89cad891de
commit 73c5d1e380
@@ -44,9 +44,19 @@ benchmark_correctness = query_db(
w = 该组错误题数
r = batch_correct_ratio
if batch_correct_ratio 为 None 或 c / (c + w) >= r:
if batch_correct_ratio 为 None:
跳过
ratio = c / (c + w)
if ratio > 1 - r:
logger.warning(
"类别 {} 正确率 {:.1%} 过高(阈值 {:.1%}),出题可能太简单",
task_type, ratio, 1 - r,
)
if ratio >= r:
跳过(正确题已充足)
k = ceil((r * w - (1 - r) * c) / (1 - r))
candidates = test_questions_dir 中同 task_type 的题
@@ -75,6 +85,7 @@ benchmark_correctness = query_db(
| DB 中无 benchmark 历史记录 | 无候选,不补入(日志警告) |
| 补入题与 test 池重复 | 允许 — test 池保持完整不动 |
| `_split_one_category` 的 floor 造成 ±1 误差 | 接受 — 比例不完美是预期行为 |
| 正确率 > 1 - r(如 r=0.5 时正确率 > 50% | `logger.warning` 警告出题可能太简单,不阻断 |
## 3. 接口变更