GCSE Maths · 40-mark half paper · benchmarked against examiner marking on 20 student scripts.
Key results:
- Mean absolute error of 0.4 marks per script after low-confidence review (0.45 marks before review), out of 40.
- 60% of scripts (12 of 20) matched the examiner’s total exactly.
- Every script (20 of 20) within one mark of the examiner’s total.
- 80 of 360 marking decisions (22.2%) flagged as low confidence for teacher review.
The paper we tested
This benchmark covers half of a GCSE Maths paper, worth 40 marks and made up of 18 separately marked sub-questions. We used 20 student scripts, which gives 360 individual marking decisions. Each script was marked by an examiner, and those marks are the definitive benchmark.
The configuration
| Exam paper | GCSE Maths |
| Total marks | 40 marks |
| Sub-questions | 18 |
| Student scripts | 20 |
| Marking decisions | 360 |
| Primary model | gemini-f-rp-3 |
| Validating model | gpt-rp-nova |
The two models come from different providers, so they are less likely to make the same mistakes. Every sub-question is marked independently by both. Where they disagree beyond the tolerance set for that question, the question is flagged as low confidence for the teacher to review. You can read more about the process in how we measure accuracy.
Results
| Measure | Result |
|---|---|
| Mean absolute error: primary model, before review | 0.45 marks (1.1%) |
| Mean absolute error: after low-confidence review | 0.4 marks (1.0%) |
| Marking decisions flagged for teacher review | 80 of 360 (22.2%) |
| Scripts matching the examiner total exactly | 12 of 20 (60%) |
| Scripts within ±1 mark of the examiner total | 20 of 20 (100%) |
Reading these figures
The mean absolute error is measured on each student’s script total. Before any review, the primary model’s total was on average 0.45 marks away from the examiner’s total, out of 40. After the low-confidence questions were reviewed, that fell to 0.4 marks.
Put another way, 12 of the 20 scripts received exactly the same total as the examiner, and none was more than one mark away.
The primary model was already very close to the examiner before review, so there was less error for the review step to remove. Around one marking decision in five was flagged. The residual 0.4 marks is not zero and we do not present it as such. It represents errors that survived both the validation comparison and teacher review.
The post-review figure assumes each flagged question is corrected to the definitive mark. It reflects the accuracy the design is built to reach once teacher review has happened.
If you would like to know how the system performs on your subject, get in touch via the contact form or book a demo below.
