What makes a response work: the information, not the grade

We are told that feedback is one of the most powerful levers in learning. That is true on average, and the average hides what matters. Before handing part of the marking to an AI, it is worth looking at what, in a response, actually produces an effect.

1.Feedback is not good by nature

The landmark meta-analysis covers 607 effect sizes, drawn from 131 articles and more than 23,000 observations (Kluger & DeNisi, 1996). The weighted mean effect is 0.41, a moderate and positive effect. But in the body of the article the authors state that more than 38% of the effect sizes are negative.

That figure has to be stated correctly, because it circulates in distorted form. It is 38% of the effect sizes, not 38% of the studies and not 38% of the interventions. It describes the spread of a distribution, not a failure rate. The article’s own abstract says “over 1/3”, which rounds down, and a well-known later meta-analysis swaps the two numbers, presenting 0.38 as the mean effect. The correct version is: a mean effect of 0.41, and more than 38% of effects negative.

What this means for a classroom is simple. Giving a response is not in itself a good pedagogical act. Some responses make performance worse.

2.What makes the difference is how much information is carried

A more recent meta-analysis, covering 435 studies and close to a thousand effect sizes, sorts responses by what they contain (Wisniewski et al., 2020). The overall effect is 0.48, and the breakdown is sharp: a response limited to reinforcing or punishing obtains 0.24, corrective feedback 0.46, and high-information feedback 0.99.

A grade sits at the bottom of that scale. It ranks, it does not inform. It tells the student where they stand without telling them anything about what to do next. The most cited synthesis in the field, which gathers twelve meta-analyses and is therefore not an original study, arrives at an effect of 0.79 for feedback in general and distinguishes four levels, according to whether the response addresses the task, the process, self-regulation or the person (Hattie & Timperley, 2007). The level that addresses the person is the least effective. A grade, which points at the student rather than at the work, sits close to it.

3.Comments or grades: what the classic study says, and what it does not

The landmark experiment assigned twelve classes of fifth- and sixth-graders to three conditions: numerical grades, individual comments, or both together (Butler, 1988). Interest and performance were highest after comments alone. Grades, and grades accompanied by comments, had similar and generally undermining effects.

That result became a slogan, and the slogan goes further than the study. A critical rereading sets out three limits (Guskey, 2019). The effect is an interaction, not a main effect: the benefit concerned students in the bottom quartile, while the strongest students who received grades kept their interest. The students in the middle, half the sample, are not addressed. And above all, the grades in the experiment were norm-referenced, based on rank in the class, while the comments addressed the task. Form and content are confounded, and we do not know which of the two produces the effect.

The defensible version is therefore narrower. Task-focused comments are better than norm-referenced grades, particularly for the students who are struggling most. It is not “stop grading”.

4.Where AI comes in, and an unexpected result

An experiment with 464 university students crossed the presence of a grade, the presence of praise, and the source attributed to the response (Lipnevich & Smith, 2008). Detailed feedback produces a large effect, and the presence of a grade lowers revision performance. One crossing is worth flagging: among the weakest students, a grade presented as coming from a computer improved revision by 3.4 points, while the same grade presented as coming from the instructor lowered it by 5.2 points.

One possible reading, which I give as a hypothesis and not as a result, is that a grade from a machine carries less social judgement. This is a single study, conducted at university, published in 2008, well before generative AI. It is not a basis for a school-level decision.

Recent studies on AI-generated feedback measure different things, and those need separating. A controlled trial with 459 upper-secondary students compares feedback produced by a language model with no feedback at all (Meyer et al., 2024): the effect on revision is 0.19, on motivation 0.36 and on positive emotions 0.34. The motivational effect is therefore larger than the effect on performance, and the comparison is not against a teacher’s feedback.

Another study had 89 student teachers rate responses without knowing where they came from (Jansen et al., 2024). The model’s feedback was judged useful for revision in 59% of texts, against 88% for an expert’s, and the raters chose to pass it on to the student in only 23% of cases. What this study measures is perceived usefulness, not learning.

Finally, a direct comparison between ChatGPT feedback and teacher feedback, with 102 university students, finds no statistically significant difference (Alsofyani & Barzanji, 2025). A null result on a modest sample weakly supports non-inferiority, and certainly not equivalence.

5.What I do with this in class

The grade is required, and it stays. The question is what travels with it.

• Separate the two moments. Give the comment first, with a chance to rework the piece, and the grade afterwards. The literature above does not set the grade against the comment, it shows that a grade placed next to a comment captures the attention.
• Give the tool the first pass on what can be checked: coverage of the criteria, structure, errors of form. Keep the judgement, which cannot be checked that way.
• Do not delegate the part that depends on the student. What this particular student should do next time is precisely what a model does not know, since it has not seen their last six pieces of work.
• Do not put students’ work into a tool as long as the canton does not provide one whose reliability is guaranteed, as the Romandie recommendations note (CIIP, 2025).

Let’s talk

How do you combine comment and grade in your classes? I would be glad to discuss it. My contact details: pablocorreaprieto.ch

Text written with the help of AI, reviewed and approved by the author.