docs(plans): add Spec-3 question-gen-v2 implementation plan
9 tasks: type extension → postprocess → families → sampler_v2 → generator_v2 → gates → run_store → pipeline_v2 → CLI integration. Includes structured-logging schemas/metrics and Codex review revisions.
This commit is contained in:
@@ -0,0 +1,33 @@
|
||||
---
|
||||
type: metric
|
||||
node_id: metric:qgen-gate-rejection-rate
|
||||
title: 出题四门总拦截率
|
||||
date: 2026-07-12
|
||||
---
|
||||
|
||||
# 出题四门总拦截率
|
||||
|
||||
## 定义
|
||||
|
||||
轻量四门(键验证 + 盲答 + 多真 + 泄漏)在首次尝试中拒绝的题目占比。
|
||||
|
||||
## 判定方式
|
||||
|
||||
- **类型**: 硬性(上限约束)
|
||||
- **阈值**: ≤ 60%(超过说明生成器质量过差,prompt 模板需大幅迭代)
|
||||
- **计算**: `COUNT(attempt=1 AND final_status='rejected') / COUNT(attempt=1)`
|
||||
- **数据源**: `question_gen_items` WHERE `attempt=1`
|
||||
|
||||
## 基线
|
||||
|
||||
待首次运行后建立。
|
||||
|
||||
## SQL
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
CAST(SUM(CASE WHEN final_status='rejected' THEN 1 ELSE 0 END) AS REAL) /
|
||||
COUNT(*) AS rejection_rate
|
||||
FROM question_gen_items
|
||||
WHERE run_id=? AND attempt=1;
|
||||
```
|
||||
Reference in New Issue
Block a user