Commit Graph

7 Commits

Author SHA1 Message Date
iomgaa cd0805aad6 chore: fix N806 lint - rename _RESAMPLE_VIDEO_INTERVAL to lowercase
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-14 06:11:01 -04:00
iomgaa 7c7ad67d81 fix(pipeline): move extra_gates after standard gate pass check
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-14 06:08:52 -04:00
iomgaa 36c712defa feat(pipeline): replace QuestionFamilySpec with TaskTypeStrategy
- SlotAssignment: remove family field, strategy looked up at process time
- PipelineConfig: remove family_ratios field
- _assign_slots: remove family_ratios and rng params (pure deterministic)
- _process_one_slot: use get_strategy() for sampling, generation, gates
- Add sub_pattern support (level/constraint override, instruction injection)
- Add strategy.extra_gates() check after standard gates
- load_pipeline_config: stop reading family_ratios from YAML
- Update tools/generate_questions.py seed override and dry-run log
- Update all integration tests to match new API

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-14 06:05:23 -04:00
iomgaa e2325b6535 refactor(sampler): replace family_spec param with level+constraint
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-14 05:38:14 -04:00
iomgaa 4fb7a61f8b fix(question_gen): resolve pipeline integration issues from final review
1. Apply postprocess shuffle result (pp.options, pp.answer) to final
   GeneratedQuestion output instead of using original candidate values.

2. Record dedup rejection in store via new mark_item_rejected() method,
   preventing items from staying as 'accepted' after dedup rejects them.

3. Add .flatten() to embed_fn outputs in _is_duplicate and embed_pool
   append to handle 2D (1,D) arrays from embedding implementations.

4. Validate exactly 4 options in _validate_parsed_fields (was >= 2),
   matching the A-D answer constraint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-12 00:14:12 -04:00
iomgaa f46e87258c refactor(question_gen): extract helpers to reduce pipeline_v2 CC below grade C
Extract _get_git_sha, _filter_pending_slots, and _run_heavy_sampling
from run_pipeline_v2. Reduces cyclomatic complexity from C(15) to B(6).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-11 23:57:35 -04:00
iomgaa 206c553143 feat(question_gen): add v2 pipeline with retry loop and heavy check
- PipelineConfig: YAML-driven configuration with family_ratios, retry,
  concurrency, dedup threshold, and heavy sampling rate
- _assign_slots: deterministic round-robin slot assignment across videos
  with per-family weighted random selection
- _process_one_slot: full retry loop (generate → postprocess → gates →
  dedup) with reject-reason feedback to VLM on retry
- _heavy_check_one: blind LLM agent trial-answer for difficulty_steps
- run_pipeline_v2: orchestration with semaphore-bounded concurrency,
  progress/resume support, and store integration
- is_duplicate: cosine similarity dedup against embedding pool

Tests: 11 integration tests covering slot assignment, retry behavior,
max-retries exhaustion, full pipeline flow, progress resume, heavy
sampling, and store record completeness.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-11 23:55:15 -04:00