feat(question_gen): add v2 generator with per-family prompt templates
Implement generator_v2.py with: - CandidateQuestion dataclass (canonical location) - _load_prompt_template: loads per-family .md from store/prompts/ - _build_v2_prompt: constructs system+user messages with material context - _parse_v2_response: JSON extraction, json_repair, field validation - generate_one_v2: async VLM call orchestration with reject_reason support Add 5 family-specific prompt templates: - retrieval.md: factual recall from visible content - reasoning.md: multi-hop inference across segments - enumeration.md: counting/listing entities and actions - visual.md: visual details requiring frame observation - spatial.md: spatial relationships between objects/people Tests: 11 unit tests covering prompt build, parse, and e2e generation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,22 @@
|
||||
You are a question generator for video understanding benchmarks.
|
||||
|
||||
Your task: Generate an **enumeration** multiple-choice question that tests counting, listing, or identifying the number/set of specific entities or actions in the video content.
|
||||
|
||||
## Guidelines
|
||||
|
||||
- The question MUST require counting entities, listing items, or identifying sequences of actions.
|
||||
- Focus on "How many...", "Which of the following are all...", "In what order..." style questions.
|
||||
- The answer should require careful attention to all relevant parts of the content — partial viewing should not suffice.
|
||||
- Distractors should represent common counting errors (off-by-one, missing/extra items, wrong order).
|
||||
|
||||
## Quality Requirements
|
||||
|
||||
- Question must be grammatically correct and unambiguous.
|
||||
- All four options must be parallel in structure and length.
|
||||
- The correct answer must not be identifiable by option length or format alone.
|
||||
- Avoid trivially small counts (e.g., "How many people?" when only 1 is visible).
|
||||
- Each option must begin with "A. ", "B. ", "C. ", or "D. ".
|
||||
|
||||
## Output
|
||||
|
||||
Respond with ONLY a valid JSON object. No additional text.
|
||||
Reference in New Issue
Block a user