Work · Case study
Marking that staysconsistent to the last script.
01 / Challenge
Marking drifted across a long pile and across markers. The same answer could earn different marks depending on who read it and how late in the day they got to it.
02 / Approach
We encoded the existing rubric as an evaluation set and built an assistant that proposes a mark and the rubric line behind it. Markers confirm, adjust or reject, and the assistant never records a mark on its own.
Agreement with human markers is measured on every release against a held-out set. A change that lowers agreement does not ship.
03 / Outcome
3×
Faster turnaround
96%
Marker agreement
Marks vary because answers vary, not because of who read them or when, and markers spend their attention on the borderline scripts.