102 lines
3.7 KiB
Markdown
102 lines
3.7 KiB
Markdown
---
|
||
type: schema
|
||
node_id: schema:question-gen-items
|
||
title: "表结构: question_gen_items（逐题门判定）"
|
||
date: 2026-07-12
|
||
---
|
||
|
||
# 表结构: question_gen_items（逐题门判定）
|
||
|
||
出题管线 v2 的逐题门判定记录。每题每轮（含重出）产生一行。
|
||
|
||
## 列定义
|
||
|
||
| 列名 | 类型 | 约束 | 说明 |
|
||
|------|------|------|------|
|
||
| `item_id` | TEXT | PK | 题目唯一 ID（UUID，每轮独立） |
|
||
| `run_id` | TEXT | NOT NULL FK→question_gen_runs | 关联批次 |
|
||
| `slot_id` | TEXT | NOT NULL | 逻辑 slot 标识（同 slot 多次重出共享） |
|
||
| `video_id` | TEXT | NOT NULL | 视频 ID |
|
||
| `family` | TEXT | NOT NULL | 题族（retrieval/reasoning/enumeration/visual/spatial） |
|
||
| `task_type` | TEXT | NOT NULL | Video-MME 12 类主标签 |
|
||
| `skill_target` | TEXT | | M1-M5 + 题族子标签 |
|
||
| `attempt` | INTEGER | NOT NULL | 当前重出轮次（1-based，≤3） |
|
||
| `gate_key_verify` | TEXT | | pass / reject / NULL（未执行） |
|
||
| `gate_blind_answer` | TEXT | | pass / reject / NULL |
|
||
| `gate_multi_true` | TEXT | | pass / reject / NULL |
|
||
| `gate_leak_test` | TEXT | | pass / reject / NULL |
|
||
| `reject_reason` | TEXT | | 拒因文本（首个拒绝门的判定摘要） |
|
||
| `final_status` | TEXT | NOT NULL | accepted / rejected / pending |
|
||
| `difficulty_steps` | INTEGER | | 重量抽检产出 Agent 步数（NULL=未抽检） |
|
||
| `selector_scores` | TEXT | | grounded selector 打分观测（JSON，见下方结构；NULL=未打分） |
|
||
| `created_at` | TEXT | NOT NULL | ISO8601 |
|
||
|
||
`selector_scores` JSON 结构：`{correct_score: float, chosen: float[], pool_size: int, anneal_rounds: int, delta_high_final: float, hard_fail: bool}`。
|
||
|
||
## DDL
|
||
|
||
```sql
|
||
CREATE TABLE IF NOT EXISTS question_gen_items (
|
||
item_id TEXT PRIMARY KEY,
|
||
run_id TEXT NOT NULL REFERENCES question_gen_runs(run_id),
|
||
slot_id TEXT NOT NULL,
|
||
video_id TEXT NOT NULL,
|
||
family TEXT NOT NULL,
|
||
task_type TEXT NOT NULL,
|
||
skill_target TEXT,
|
||
attempt INTEGER NOT NULL DEFAULT 1,
|
||
gate_key_verify TEXT,
|
||
gate_blind_answer TEXT,
|
||
gate_multi_true TEXT,
|
||
gate_leak_test TEXT,
|
||
reject_reason TEXT,
|
||
final_status TEXT NOT NULL DEFAULT 'pending',
|
||
difficulty_steps INTEGER,
|
||
selector_scores TEXT,
|
||
created_at TEXT NOT NULL
|
||
);
|
||
|
||
CREATE INDEX idx_items_run ON question_gen_items(run_id);
|
||
CREATE INDEX idx_items_family ON question_gen_items(family);
|
||
CREATE INDEX idx_items_task_type ON question_gen_items(task_type);
|
||
CREATE INDEX idx_items_slot ON question_gen_items(slot_id);
|
||
```
|
||
|
||
## 埋点位置
|
||
|
||
| 代码位置 | 动作 | 时机 |
|
||
|---------|------|------|
|
||
| `synthesizer.py::generate_one_v2` 返回后 | INSERT（gates=NULL, status=pending） | 每题生成后 |
|
||
| `gates.py::run_gates` 返回后 | UPDATE gate_* 列 + reject_reason + final_status | 门跑完后 |
|
||
| 重出循环入口 | INSERT 新行（same slot_id, attempt+1） | 重出时 |
|
||
| `heavy_check.py` 完成 | UPDATE difficulty_steps | 抽检完成 |
|
||
|
||
## 查询模式
|
||
|
||
```sql
|
||
-- 各门拦截率
|
||
SELECT
|
||
gate_key_verify,
|
||
COUNT(*) as cnt
|
||
FROM question_gen_items WHERE run_id=?
|
||
GROUP BY gate_key_verify;
|
||
|
||
-- 按题族切片拦截率
|
||
SELECT family,
|
||
SUM(CASE WHEN final_status='rejected' THEN 1 ELSE 0 END) as rejected,
|
||
COUNT(*) as total
|
||
FROM question_gen_items WHERE run_id=?
|
||
GROUP BY family;
|
||
|
||
-- 重出收敛率
|
||
SELECT
|
||
COUNT(DISTINCT slot_id) FILTER (WHERE final_status='accepted') as converged,
|
||
COUNT(DISTINCT slot_id) as total_slots
|
||
FROM question_gen_items WHERE run_id=?;
|
||
|
||
-- 按 task_type 分布
|
||
SELECT task_type, COUNT(*) FROM question_gen_items
|
||
WHERE run_id=? AND final_status='accepted'
|
||
GROUP BY task_type;
|
||
```
|