iomgaa
271d1682c9
Implement 4 concurrent LLM-based quality gates for generated questions: - key_verify: validates answer evidence in source material - blind_answer: rejects questions answerable without video context - multi_true: detects ambiguous multi-correct options - leak_test: per-family shortcut detection (5 probe templates) Includes run_gates orchestrator with verbatim_ratio short-circuit, JSON response parsing with fallback, and 9 unit tests (all passing). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
688 B
688 B
Multi-True Gate
You are checking whether a multiple-choice question has more than one plausibly correct answer given the source material.
Source Material
{source_text}
Question
{question}
Options
{options}
Instructions
- Read the source material and the question carefully.
- For each option, assess whether it could be considered a correct or plausible answer given the source.
- If exactly ONE option is clearly correct → verdict "pass".
- If TWO or more options are plausibly correct → verdict "fail".
Response Format (strict JSON)
{{"verdict": "pass" or "fail", "reason": "brief explanation listing plausible options if multiple"}}