These three blogs (part I, part II here, part III) are based on a talk I gave at Headteachers’ Roundtable Summit in March 2019. My thoughts on this topic have been extensively shaped by conversations with Ben White, a psychology teacher in Kent. Neither of us yet know what we think!

The two fundamental jobs that children need to do are to feel successful and to have friends – every day. Sure, they could hire school to get these jobs done. Some achieve success and friends in the classroom, the band, the math club, or the basketball team. But to feel successful and have friends, they could also drop out of school and join a gang, or buy a car and cruise the streets. Viewed from the perspective of jobs, it becomes very clear that schools don’t often do these jobs well at all – in fact, all too often, schools are structured to help most students feel like failures.

Clayton M. Christensen, James Allworth and Karen Dillon
How Will You Measure Your Life?

Where do we believe we are on the grading curve?

Whether we like it or not, from a very young age students start to develop an idea of where they sit on a bell curve of attainment, relative to their peers. All that cohort-referenced feedback does is to give students new information about how well they are doing in the ‘game’ of trying to climb up the bell curve. If only it were so simple that teachers could encourage students to make an effort in this ‘game’ by handing out regular cohort-referenced feedback! Though the examples in my first post showed that it’s a good bet that introducing cohort-referencing will raise effort, on average, other studies show it is risky strategy because it can alter beliefs in quite unhelpful ways.

How can this happen? Surely everyone wants to work hard to get as far up the ladder as they can? This blog post sets out the mental model that explains three dimensions of each of your student’s belief systems that you must consider when thinking through how they respond to learning their attainment.

Beliefs about where we sit on the curve

There is pretty convincing evidence that, rather than constantly striving to be the best, we instead tend to prioritise self-esteem maintenance or keeping ourselves at the place in the hierarchy to which we have become accustom! (Remember we are having to juggle multiple games in life so choice about where to direct effort is sometimes necessary.) This means that the effect of attainment feedback rather depends on how it allows us to update our prior ideas about position on the bell curve (or at least lessen the fuzziness we have about how well we are doing).

Where a mark or grade received simply confirms our prior view of how well we were doing, the act of receiving the feedback might have no impact because we have no need to adjust behaviours to close the gap between our current performance and the internal standard we had in mind for ourselves.

This was demonstrated in a nice experiment, albeit one on undergraduates, which provided students with their position in the grade distribution every six months. The researchers found that giving this grading feedback actually LOWERED academic outcomes more frequently than it raised them! How can this be? Well, it turns out that, in the absence of knowing their rank, many of these students had actually underestimated how well they were doing. When they received the good news they were doing better than they thought, their (self-reported) satisfaction increased and they ramped down the effort they were putting into their studies. There was a smaller group for whom the reverse was true – they had overestimated their position in the year and so receiving the bad news of their true ranking caused them to increase their effort.

England’s long experiment with giving Year 12 students additional nationally-benchmarked feedback in the form of AS exam results was mirrored by a nicely-evaluated Greek policy experiment that introduced a nationally-benchmarked penultimate year exam. The strongest predictor of your response to this type of additional penultimate year feedback was whether it gave you a positive or negative surprise.

• Negative surprise: Learning that your 11^th grade performance was worse than for other students with a similar 10^th grade scores to you generally led to greater effort for final year exams, regardless of your prior attainment (shown in green).
• Positive surprise: Learning you did better in 11^th grade than would be suggested by your 10^th grade position led to less effort for final year exams (red on the chart).

This finding that prior over or under confidence in attainment is central to how we respond to feedback is an incredibly important consideration when trying to second-guess how your students might respond to the feedback you give them. If they find out they are doing worse than they thought, they tend to pull your finger out! If they find out they are doing better than they thought, they tend to step off the gas! It is prior beliefs in attainment, relative to actual attainment, that frequently predict a student’s response (and this finding is replicated in many studies).

Now, it’s worth saying that, although this wasn’t true for the university grading study described above, people tend to be a little over-optimistic about how well they are doing [£] so giving clearer feedback tends to be more helpful than not in pushing people to make an effort. However, this doesn’t mean that getting bad news is ALWAYS useful. To understand why, we need to look at how feedback shapes other beliefs we hold about ourselves.

Beliefs about productivity of effort

So far, we’ve learnt how feedback shapes the beliefs we hold about our current performance. However, it also shapes our beliefs about our ability to learn or climb up the rankings. Receiving a grade is small part of a feedback loop, all the pieces of which must be in place for it to produce a useful behavioural response. The student needs to believe it is possible to play the game, i.e. that their effort can productively translate into learning that in turn translates into performance in the next test. (Where students do not feel they have the knowledge and capacity to translate effort into learning then decisions about how to convey attainment are irrelevant.)

All teachers know how important it is for students to maintain self-efficacy in order to persevere over extended periods of time at school, often without short-term rewards. Teachers will also know that, whilst we can be sure that grit and growth mindset are malleable characteristics, particularly in young children, it has proved difficult to construct reliable interventions to improve mindset. Unfortunately for us in designing grading feedback, it can often simultaneously influence beliefs in performance and beliefs in ability to learn such that they work against each other. Learning you’ve been unexpectedly successful in a test enforces a belief in your ability to learn, yet dampens your desire to do so!

Feedback can, correctly or incorrectly, seriously impair our belief in the productivity of our effort – what Dweck (1986) calls ‘learned helplessness’. So, you’ve found out you are towards the bottom of the bell curve. You’ve come to believe that even a huge amount of effort can’t get you to a much better position. Then what? Well, you might be OK with reducing your own internal standard of how good you want to be and continue to take part in the grading game…. but it’s understandable that it doesn’t appeal to everyone.

Choosing to play your grading game

Young dropouts are often behaving quite as rationally as their more successful peers. If you are fourteen or fifteen and well behind academically, your chances of catching up start to look very very slim. And if you are not going to improve your relative position much, even if you change your whole lifestyle, then why not give up on the academic competition completely? It is likely to be better for your self-respect and, at least in the short term, not obviously worse for your other prospects: time spent in school when it isn’t going to improve your job opportunities, and you don’t enjoy the work, is just time wasted.

Alison Wolf
Does Education Matter?

Nobody likes to compete in a game they know they will do badly in, regardless of how hard they work. This is particularly true in education, where our ability can be tightly intertwined with our sense of self and how other people perceive us. Empirical research shows that in addition to liking to be at the top of the pile, we are also strongly “last-place” averse. If avoiding the bottom of the ladder is a powerful motivator in humans, how are your students going to try to avoid it? By trying to climb up? Or by trying to climb off?

Even where performance in the game is kept entirely private, there is good evidence that students develop strategies to avoid learning their ranking. In experiments, those who suspect they are lower performers are more likely to avoid learning their rank in class (e.g. here), thus refusing to take part in the grading game. Even worse is the phenomenon of ‘self-handicapping’ – deliberately withdrawing effort where there is potential for self-image-damaging feedback. For example, staying up late gaming with friends before an important exam allows us to attribute poor exam performance to the tiredness rather than low ability.

Where rankings are made public, our responses to them will be strongly shaped by adherence to social norms and the preservation of our social reputation. I don’t want to get into reviewing the literature on award-giving (here is a nice summary), but it makes it clear that responses are heavily framed by peer context. Awards can be motivating if handed out with care in the right circumstances (i.e. sparingly, in just-circumstances, exploiting the bond of loyalty between giver and receiver, but with consideration of effects on non-recipients). However, if we choose to make academic effort observable to peers by publishing grades or awards, students may act to avoid social penalties or gain social favour by conforming to prevailing norms.

In the US, academics Bursztyn and Jensen have run school experiments highlighting how these risks play out with students. They introduced a performance-based leaderboard for an online learning platform which announced the top three performers in the classroom, in the school, and overall on the platform. This publication of the leaderboard led to a decrease in student performance on the platform, which was primarily driven by a decline in effort provided by students who were top performers prior to the introduction of the leaderboard. This suggests it created a fear of peer social sanction for high performers (by contrast, if anything, the performance of those at the bottom of the distribution slightly improved).

In another experiment, they showed that whether or not you sign-up to a voluntary test preparation course is affected by whether your sign-up is made public. In classrooms with high attaining students taking advanced classes, sign-up rates were unaffected by whether the enrolment decision was public. It was in the classrooms with less high attaining students that enrolment rates were much lower with a public sign-up system. Social norms of the classroom matter. (A nice feature of the study was their ability to observe the same student sitting alongside high achieving peers in one classroom and alongside more mixed peers in another – think about a scenario where a student is in ‘top set’ for one subject and mixed ability class for another. These same students were negatively affected by the sign-up sheets being made public in the more mixed class but were not in the high-attaining class.)

The mental model of responses to grades for your students

Handing out clear, cohort or nationally-referenced feedback to students or their parents on how they are performing has the potential to be a powerful motivator, but comes with risks. Knowing a student’s ability or attainment cannot help you predict how they will respond. What matters is how the grading information:

• Changes their beliefs about their attainment
• Changes their beliefs about their ability to learn and get better
• Changes their desire to keep playing the competition of trying to be the best, or maintain their position, or avoid the bottom rung

Every teacher has to give some feedback on attainment and there is no risk-free or value-neutral approach to doing it. I hope you can think about the mental model I’ve described in relation to students you know. Have you ever asked them, for example, how well they think they are doing before you hand out their grades? Who is suffering from ‘learned helplessness‘ and is there anything you can do to counter that? Which five students you teach are most likely to respond negatively to learning their attainment in a more transparent way?

Schools do quite bizarre things to avoid telling students their position on the attainment bell curve. Bizarre, yes, but not necessarily unhelpful. In the final blog post, I will use this mental model to describe why schools end up creating these strange grading games!

You can find part III here.