AQA GCSE English Language Paper 1 (8700/1) · 40-mark question · benchmarked against an external examiner on 50 real student responses.
Key results:
- Mean absolute error of 2.18 marks after review (3.08 marks before review).
- 32% of responses (16 of 50) flagged as low confidence for teacher review.
The question we tested
This is Question 5 of the AQA GCSE English Language Paper 1 (8700/1), a real exam question worth 40 marks. It asks students to produce an extended piece of creative writing, either a description or a narrative, assessed for content and organisation (24 marks) and technical accuracy (16 marks) (AO5 and AO6). Question 5 is the single largest and most demanding piece of extended writing on the paper, and the task markers find hardest to assess consistently.
How Running Paper marks this question
Running Paper marks pen-and-paper assessments using two independent AI models. A primary model marks every response. A second, independent model marks it again as a validator, and the two never see each other’s marks. Where they disagree by more than a set tolerance, the response is flagged as low confidence and routed to the teacher to review, with the mark scheme and the response to hand. The teacher stays in control of every mark that matters.
For this question: the primary marker is gemini-rp-4 and the validator is gpt-rp-3. A response is flagged for review when the two models’ marks differ by 20% of the available marks or by 6 marks, whichever is smaller.
You can read more about the process in how we measure accuracy.
How we measured accuracy
We took 50 real student responses and had them marked independently by an external examiner. These are the definitive benchmark marks. We then marked the same responses with Running Paper, with no access to the benchmark, and compared the two. The figures below use three measures:
- Mean absolute error: how far Running Paper’s marks are from the definitive benchmark marks on average.
- Percentage within tolerance: how often Running Paper’s mark for a response falls within a given number of marks of the definitive benchmark mark.
- Percentage of questions flagged: how often the primary and validating models disagree enough to trigger a low-confidence review.
Results
| Figure | Value |
|---|---|
| Mean absolute error (before review) | 3.08 marks (7.7%) |
| Mean absolute error (after review) | 2.18 marks (5.5%) |
| Questions flagged for review | 32% (16 of 50) |
Percentage within tolerance
How many responses Running Paper marked within each number of marks of the benchmark, before and after review:
| Within | Model only | After teacher review |
|---|---|---|
| ±1 mark | 14 of 50 (28%) | 22 of 50 (44%) |
| ±2 marks | 18 of 50 (36%) | 26 of 50 (52%) |
| ±3 marks | 28 of 50 (56%) | 34 of 50 (68%) |
| ±4 marks | 40 of 50 (80%) | 43 of 50 (86%) |
| ±5 marks | 43 of 50 (86%) | 46 of 50 (92%) |
| ±6 marks | 48 of 50 (96%) | 50 of 50 (100%) |
Reading these figures
This is where the validation process does the most work. The two models disagreed on almost a third of responses, and reviewing those cut the mean absolute error from 3.08 to 2.18 marks, which is 5.5% of the 40 marks available. After review, every response was within six marks of the examiner.
If you would like to know how the system performs on your subject, get in touch via the contact form or book a demo below.
