AQA GCSE English Language Paper 1 (8700/1) · 8-mark question · benchmarked against an external examiner on 50 real student responses.
Key results:
- Mean absolute error of 0.44 marks after review (0.62 marks before review).
- 20% of responses (10 of 50) flagged as low confidence for teacher review.
- Every response within 2 marks of the examiner’s mark after review.
The question we tested
This is Question 2 of the AQA GCSE English Language Paper 1 (8700/1), a real exam question worth 8 marks. It asks students to analyse how the writer uses language to describe a scene (AO2), looking at words, phrases and language techniques in a short extract. Short, criteria-driven analysis like this is where consistent marking matters most, because every mark is a large share of the total.
How Running Paper marks this question
Running Paper marks pen-and-paper assessments using two independent AI models. A primary model marks every response. A second, independent model marks it again as a validator, and the two never see each other’s marks. Where they disagree by more than a set tolerance, the response is flagged as low confidence and routed to the teacher to review, with the mark scheme and the response to hand. The teacher stays in control of every mark that matters.
For this question: the primary marker is gemini-rp-4 and the validator is gpt-rp-3. A response is flagged for review when the two models’ marks differ by 20% of the available marks.
You can read more about the process in how we measure accuracy.
How we measured accuracy
We took 50 real student responses and had them marked independently by an external AQA examiner. These are the definitive benchmark marks. We then marked the same responses with Running Paper, with no access to the benchmark, and compared the two. The figures below use three measures:
- Mean absolute error: how far Running Paper’s marks are from the definitive benchmark marks on average.
- Percentage within tolerance: how often Running Paper’s mark for a response falls within a given number of marks of the definitive benchmark mark.
- Percentage of questions flagged: how often the primary and validating models disagree enough to trigger a low-confidence review.
Results
| Figure | Value |
|---|---|
| Mean absolute error (before review) | 0.62 marks (7.8%) |
| Mean absolute error (after review) | 0.44 marks (5.5%) |
| Questions flagged for review | 20% (10 of 50) |
Percentage within tolerance
How many responses Running Paper marked within each number of marks of the benchmark, before and after review:
| Within | Model only | After teacher review |
|---|---|---|
| ±1 mark | 47 of 50 (94%) | 48 of 50 (96%) |
| ±2 marks | 50 of 50 (100%) | 50 of 50 (100%) |
Reading these figures
Running Paper was already close to the examiner on this question before any review: 94% of responses were within one mark and every response was within two. One response in five was flagged as low confidence, and reviewing those brought the mean absolute error down from 0.62 to 0.44 marks.
If you would like to know how the system performs on your subject, get in touch via the contact form or book a demo below.
