docs: add high-correctness warning to maintenance pool design
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -44,9 +44,19 @@ benchmark_correctness = query_db(
|
||||
w = 该组错误题数
|
||||
r = batch_correct_ratio
|
||||
|
||||
if batch_correct_ratio 为 None 或 c / (c + w) >= r:
|
||||
if batch_correct_ratio 为 None:
|
||||
跳过
|
||||
|
||||
ratio = c / (c + w)
|
||||
if ratio > 1 - r:
|
||||
logger.warning(
|
||||
"类别 {} 正确率 {:.1%} 过高(阈值 {:.1%}),出题可能太简单",
|
||||
task_type, ratio, 1 - r,
|
||||
)
|
||||
|
||||
if ratio >= r:
|
||||
跳过(正确题已充足)
|
||||
|
||||
k = ceil((r * w - (1 - r) * c) / (1 - r))
|
||||
|
||||
candidates = test_questions_dir 中同 task_type 的题
|
||||
@@ -75,6 +85,7 @@ benchmark_correctness = query_db(
|
||||
| DB 中无 benchmark 历史记录 | 无候选,不补入(日志警告) |
|
||||
| 补入题与 test 池重复 | 允许 — test 池保持完整不动 |
|
||||
| `_split_one_category` 的 floor 造成 ±1 误差 | 接受 — 比例不完美是预期行为 |
|
||||
| 正确率 > 1 - r(如 r=0.5 时正确率 > 50%) | `logger.warning` 警告出题可能太简单,不阻断 |
|
||||
|
||||
## 3. 接口变更
|
||||
|
||||
|
||||
Reference in New Issue
Block a user