EngageNY ELA
New York State (EngageNY)
Judge anthropic/claude-opus-4-7 · Signal Studio judge (claude-opus-4-7) · 2026-05-28
% gateway
exact match
Agreement vs gold
Gateway level — gold and the judge as two raters over the three gateways.
Exact match
100.0%
Weighted κ
1.00
MAE
0.00
Signed bias
+0.00
Indicator-level gold isn’t available for this curriculum (EdReports’ published ELA report exposes gateway ratings only), so accuracy is measured at gateway grain. Indicator-grain gold is a pending data-acquisition task.
Attribution
Where the judge diverges, and whether it's signal: per-gateway bias localizes the gap, the confusion matrix shows over- vs under-rating, and the noise floor is the judge's self-agreement across passes (a gap below it isn't trustworthy).
Gateway rollup
Indicator scores rolled up to EdReports gateway ratings (sequential gating + no-0s cap), gold vs judge.
G1 · Text Quality and Complexity
26/28 pts
G2 · Building Knowledge
32/32 pts
G3 · Usability
23/25 pts
Gateway-level agreement: exact 100% · κ 1.00 (3 gateways)
Divergences (0)
Indicators where the judge's score differed from gold. Amber = judge under-scored, blue = over-scored.
No divergences
Judge and gold agreed on every scored indicator.