feat(eval): classification evaluator schemas + sample projects + e2e tests#1663

Draft

ajay-kesavan wants to merge 19 commits into

mainfrom

feat/classification-evaluator-types

Commits on Jun 19, 2026

Merge remote-tracking branch 'origin/main' into feat/eval-dataset-evaluators
ajay-kesavan
committed
Merge remote-tracking branch 'origin/main' into feat/classification-evaluator-types
ajay-kesavan
committed
refactor(eval): embed aggregator specs in per-datapoint evaluator configs

ajay-kesavan
and
claude
committed
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types
ajay-kesavan
committed
feat(eval): wire sample classification evaluators to embedded aggregators

ajay-kesavan
and
claude
committed
refactor(eval): apply ponytail-review cleanup

ajay-kesavan
and
claude
committed
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types
ajay-kesavan
committed
refactor(eval): apply ponytail-review cleanup (justification + demo)

ajay-kesavan
and
claude
committed
fix(eval): address adversarial-review feedback on dataset evaluators

ajay-kesavan
and
claude
committed
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types
ajay-kesavan
committed
fix(eval): address adversarial-review feedback on classification samples

ajay-kesavan
and
claude
committed
fix(eval): address codex P1 + lint failures on dataset evaluators

ajay-kesavan
and
claude
committed
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types
ajay-kesavan
committed
test(eval): drop fscore-duplicate test that conflicts with #1663 H2 validator

ajay-kesavan
and
claude
committed
fix(eval): publish aggregators in classification evaluator type schemas

ajay-kesavan
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(eval): classification evaluator schemas + sample projects + e2e tests#1663

feat(eval): classification evaluator schemas + sample projects + e2e tests#1663
ajay-kesavan wants to merge 19 commits into
mainfrom
feat/classification-evaluator-types

Commits on May 20, 2026

feat(eval): add evaluator type schemas for classification evaluators

test(eval): add e2e tests + sample projects for classification evaluators

feat(eval): add dataset-level evaluator framework with precision/recall/f-score

docs(eval): add runnable dataset evaluator demo + bump uv.lock for 2.10.69

Commits on Jun 19, 2026

Merge remote-tracking branch 'origin/main' into feat/eval-dataset-evaluators

Merge remote-tracking branch 'origin/main' into feat/classification-evaluator-types

refactor(eval): embed aggregator specs in per-datapoint evaluator configs

Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types

feat(eval): wire sample classification evaluators to embedded aggregators

refactor(eval): apply ponytail-review cleanup

Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types

refactor(eval): apply ponytail-review cleanup (justification + demo)

fix(eval): address adversarial-review feedback on dataset evaluators

Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types

fix(eval): address adversarial-review feedback on classification samples

fix(eval): address codex P1 + lint failures on dataset evaluators

Merge branch 'feat/eval-dataset-evaluators' into feat/classification-evaluator-types

test(eval): drop fscore-duplicate test that conflicts with #1663 H2 validator

fix(eval): publish aggregators in classification evaluator type schemas