Commit Graph

4 Commits

Author SHA1 Message Date
iomgaa 25918a73ff fix: normalize non-scalar prediction; harden predictions insert 2026-07-16 06:06:58 -04:00
iomgaa 8a69a54078 refactor: fail-loud unit grading and stricter orphan pair drop
Address review: replace bare KeyError with a contextual ValueError invariant
check in _grade_unit (fail-loud, no catch/skip/fallback); tighten
_drop_orphan_pairs to require exactly one original + one mirror with no extra
illegal-role records (total==2); clarify InferenceResult docstring to unit-grained
semantics. Add tests for missing-prediction descriptive error and extra-illegal-role
pair drop.
2026-07-15 07:05:37 -04:00
iomgaa 730caa7e9a feat: aggregate inference by question unit with pair AND
Reuse build_units/unit_correctness (pair contract single entry) in the
inference aggregation step: single questions count as one unit, AR pairs
collapse original+mirror into one unit scored by bidirectional AND. total/
correct/per_task_type are unit-grained; orphan pairs (missing one side) are
warned and dropped, not counted. Per-question predictions still land row by
row (traceability unchanged).
2026-07-15 06:57:48 -04:00
iomgaa 6b56fc8f98 feat(harness): inference.py — async run_inference + DI 2026-07-07 13:04:26 -04:00