It’s not (just) what teachers know, it’s who teachers know

I have been talking to many teachers and school leaders recently about what information needs to be recorded, whether in a markbook or in a centralised system, for a teacher to teach effectively. The answer is, partly, that it depends on what information the teacher is able to hold in their head, without the need for taking notes! A primary school teacher who spends 25 hours a week with the same 30 children has a rather easier job here than the secondary school music teacher who sees over 300 students pass through their classroom each week.

I recently dropped in to see a school where the need for written documentation was about as low as it is possible for it to be: a one-form entry primary school, very low family mobility, stable teaching staff, a headteacher that can know all the students and their parents by name. We spoke for some time about what he thinks his teachers need to ‘know’ about students to do their job – the importance of nuanced views of what a child can already do, how difficult students are likely to find a new task they encounter and how best to engage the child in learning. We spoke about how he thinks teachers accumulate this information about their students over the course of a year, and what is lost when students move to a new class in September.

He then mentioned in passing that they had decided to keep a number of classes with the same teacher last September. ‘So powerful to start the year already knowing your students!’, he said. In reply, I told him about a recent US research study that backs up his intuition. American educators call the policy of keeping students with the same teacher for a second academic year looping – what a great phrase! The study (£) in elementary schools showed small academic gains from keeping students with the same teacher for a second year. It is important to note that the effect size here isn’t massive, but in education policy we are almost always in the business of marginal gains.

Of course, the popularity of looping rather depends on having a pool of consistently effective teachers. In my family, we often still talk about one person’s disastrous three-year ‘looping’ experience as an infant pupil with an ineffective teacher. Looping practice for up to eight years in Steiner (Waldorf) Schools is said to lead to parents removing their children en masse from one class if they aren’t happy with their teacher.

Studies on the benefits of looping serve to remind us about the importance of teachers knowing their classes. Secondary schools make an effort to loop in years 10 and 11, but perhaps they should seek to extend this looping back into the younger years too. Other practices, such as ensuring form tutors also get to teach their classes or minimising incidences of ‘split’ classes, would seem to be important too but are increasingly hard to achieve where tight budgets leave no flexibility in timetabling arrangements. More controversially, it highlights one difficulty with job share arrangements in primary schools, where the part-time teachers necessarily take longer to get to know their class at the start of the year.

Other commentators have rightly drawn parallels with another study where elementary school teachers specialised in (usually) two of maths, science, English or social science and taught these subjects across multiple classes. The effect of this subject specialism was to lower pupil achievement. The author reported that “… teacher specialization, if anything, decreases student achievement, decreases student attendance, and increases student behavioural problems.”

Now, this wasn’t proper subject specialism that included training to become specialists: headteachers at each school simply helped to identify who should be allocated to specialise in which subject. That said, I interpret this second study as showing that we might want to think again about the trade-offs between having teachers who are subject experts, able to benefit from both disciplinary expertise and repeating the same lessons, and teachers who are experts in the students they teach. It is inevitably hard to be an expert in both.

In England, our schooling careers are U-shaped with respect to whether teachers know us well or not. Our youngest and oldest students benefit from a few teachers who get to know them very well. By contrast, between the ages of 11 and 14, students troop between a dozen different teachers each week. Are we sure we always get the trade-offs right? For example, how did we decide it was optimal to have one generalist teacher for ten year-olds, followed by ten subject specialists for eleven year-olds? Did middle schools who chose to run a core part of each teaching day with the form teacher get something right? And whilst history teachers love not having to teach religious studies and physics teachers love not having to teach biology, how far should a school fractionalise a pupil’s timetable before it becomes damaging to their academic and pastoral experience? These are empirical questions that we cannot yet answer.

It is great that so much policy energy has been focused on a more sophisticated understanding of the curriculum, of what makes subject knowledge domains distinctive and of what this implies for subject-specialist pedagogy. We should harness this sophisticated understanding of the number of hours it would take to train a teacher to deliver a particular curriculum, to a particular age group, with resources we may or may not have provided for them, to think deeply about whether we’ve always got the trade-offs right between becoming specialists in subjects or specialists in children.

If CPD is so important, then why is so much of it so bad?

Towards the end of last year I took part in a debate about the quality of CPD. I was asked to take one side of the argument, so this is my deliberately one-sided perspective on it. The wonderful people of edu-twitter helped me compile the bizarre examples of CPD that you’ll read below.

Everybody remembers their worst ever INSET day, don’t they? I remember being excited to take part in one early on in my PGCE. This was in the early 2000s, so the National Strategies were landing in Key Stage 3. The Local Education Authority’s maths officer came to the school to help every teacher embed numeracy in the Key Stage 3 curriculum. It was a strange morning for us, as a department (business/economics) who taught no Key Stage 3 and yet weren’t allowed to get on with other things. It was also a strange morning for the maths department, who presumably had already embedded numeracy in their curriculum. And I’d guess at least half the other departments were pretty annoyed to have to think of a contrived way to mention numbers during their lessons. Whose fault was it that so few teachers got anything out of that day? Should the headteacher have allowed it to happen? Should the LEA have known better than to blithely follow DfE ideas about numeracy at KS3? And where was teacher agency in the decision about allocating INSET time to this?

I asked teachers on twitter to name their worst ever CPD, and it quickly became clear they had examples far worse than mine:

It was one which postulated the theory that we have 5 brains, one of which developed when humans were running away from dinosaurs. I kid you not.— David Williams (@davowillz) October 20, 2018

A day on packtypes. We all answered questions about ourselves and others to find out what dog we were. Then we made bunting to go around school to show our packtypes. I have since read the research behind it but the watered down approach did not communicate anything useful— Mrs Cadman (@Y6Mrs) October 20, 2018

The worst I can recall was about 25 years ago at least on ‘Instrumental Enrichment’. I had no idea what it meant and even less after the CPD.— Gradgrind (@ThomGradgrind) October 20, 2018

It was something to do with animals representing different types of learners. The CPD was ten minutes and we were then given a week’s deadline to ’embed the animals in schemes of learning’.— Deborah Hawkins. (@debbs_198) October 20, 2018

In groups, we had to spend an hour making a 2 minute, creative video about one of the schools values. It was shown to the other staff on the day, but no discernible purpose – even when I asked.— Stuart Garner (@SJGarner76) October 20, 2018

All of the PD events on questioning… Where no actual questioning takes place. And instead mind numbing droning for 1-2 hours. #HowNotToTeach— Mr. OatesSoSimple (@MrOatesSoSimple) October 20, 2018

Now twitter trades on anecdotes so let’s turn to a slightly larger pool of teacher opinion on Teacher Tapp. Less than a third of classroom teachers agree with the following statement: ‘Time and resources allocated to professional development are used in ways that enhance teachers’ instructional capabilities.’ (Of course, the vast majority of senior leaders agreed with the statement!) We consistently see that it is secondary school classroom teachers that are the most negative about their experiences of CPD, perhaps because so little is subject specific. For example, 40% of them feel that CPD has had little or NO impact on their classroom practice. Many classroom teachers also report that abandoning INSET days would have no effect on their teaching!

This presents us with a serious question to answer: If professional development is so important, then why is so much CPD so bad?

I think it is reasonable to suggest that the people who commission training, particularly in secondary schools, either don’t know how to find good provision or are severely resource constrained, which in turn leads them to deliver quite generic whole school training.

However, I’d like us to consider an alternative explanation. That is that ‘we’, as a profession, don’t really know how to do CPD. This reminds me of the arguments as to why the NHS does not fund greater mental health provision. Of course it is, in part, because they lack the funds. But it is also because we haven’t worked out any scaleable cost-effective means of treating it yet. Just as doctors don’t reliably know how to make unhappy people happier, education professionals don’t reliably know how to make teachers more effective.

One promising vein of research has been into instructional coaching, where a number of randomised controlled trials have estimated positive programme effects. Does this mean all teachers should have instructional coaches who are trained in how to use the observational rubrics to give feedback? No. Coaching is a great, but expensive, way to get teachers from being not OK at teaching to being good. Simple rubrics can’t support coaches in taking teachers from good to great because this is likely to be highly sensitive to the curriculum and demographics of students.

Moreover, even for inexperienced teachers, the expense means we have to ration the treatment. As with one-to-one therapy or tuition, we withhold it from most teachers and hope they figure out how to get better through a cheaper method – such as reading books or trial and error.

We have some of the best brains in the education system thinking about how best to spend CPD money (e.g. David Weston, the people at Institute for Teaching). However, I wonder whether there are even more important things to worry about first. I’d start with worrying about whether schools can provide teachers with a stable curriculum, with assignment to appropriate classes, with a healthy approach to workload to give teachers the space to think about their teaching, and of course a culture where teachers can teach and are encouraged to get better at teaching on their own.

When I think of the incredible teachers I’ve watched over the past few years, I wonder how they got so good at teaching. When I’ve had the opportunity to ask them, they have never volunteered that formal professional development courses made a material contribution. Of course, these teachers who made it to great without losing morale or leaving first are, alas, the exception. But what makes us think that the courses they didn’t use to successfully get better at teaching will be helpful to others who want to get there too?!?

I think that supporting teachers in getting better at teaching is critical to teacher morale and the long-term health of schooling system. This is the central thesis of our book, The Teacher Gap.

I just don’t know whether we’ve found our medicine.

It was fun arguing with people from across the world about how we get professional development right. Somebody on twitter pointed me to this gem of a quote that will always make me smile:

I hope I die during an in-service session because the transition between life and death would be so subtle.

Helen Timperley (2011) quoting an anonymous teacher

How can we learn if Teach First is working?

Last week I published a paper I wrote with Jay Allnutt about the impact of Teach First on GCSE attainment. We received a large amount of feedback on the paper, via a seminar presentation at BERA conference, comments on a blog I wrote, twitter and email. Rather than simply present these research findings at researched2013, I showed the audience some of feedback we received on the paper from the education community to reflect on how research can move forward understanding of major education policies. This blog gives an approximate overview of my presentation at researched2013, following on from Joe Kirby’s @joe__kirby blog of his talk.

The paper I published with Jay Allnutt showed that schools participating in Teach First improved their GCSE results. In their second year of participation, the improvements were in the order of one grade in one of a pupil’s best eight subjects (5% of a pupil s.d.) or a two percentage point gain in the % achieving five or more GCSEs at A*-C (incl. Eng and maths).

Making these sorts of claims was not straightforward because…

…Schools were not selected at random to join Teach First

I think all new education policies should be randomised in their implementation, providing it is possible. Teach First received substantial government funds so has a duty to taxpayers to demonstrate its effectiveness. This was a major failing on the part of Civil Servants who agreed to fund it (Sam Freedman @samfr spoke eloquently about this issue). Rather than recruit school’s from 20 local authorities in London in the first year of the programme, recruitment should have taken place across a randomly drawn lot of deprived schools or alternatively local authorities should have been randomised into the programme.

This did not happen, producing a very serious identification problem because Teach First schools look very different to others – they are concentrated in London, are relatively deprived, were likely to have had particularly severe teacher recruitment problems and may have had headteachers who were particularly dynamic or risk-taking.

We try to deal with non-random selection in our estimation by finding a set of schools that look identical to the Teach First schools, except that they were not participating in the early years:

Is Teach First working

And rather than simply comparing GCSE performance between our Teach First schools and a matched control group of schools, we run regressions that model changes taking place year-by-year at every school:

Is Teach First workingDID

The first set of responses we received to our papers asked…

Are there confounding factors?

“it seems possible that managerial teams that are ‘early adopters’ [of Teach First] are a different calibre to ‘followers’ who catch up with trends after it starts to look cool … [so it might be] … higher managerial chutzpah of the early adopters doing others things nothing to do with TF that cause results to rise”

[blog comment]

For this type of criticism to be valid, the timing of the adoption of ‘higher managerial chutzpah’ must have to exactly coincide with the participation in Teach First. (Why? Well, we match on change in GCSE scores so their superior performance cannot have preceded the programme and we perform falsification tests, which demonstrate that Teach First did not have an impact in the years before the school joined the programme.) But more importantly, if our findings are entirely due to ‘higher managerial chutzpah then we would not witness positive effects of Teach First for those departments who received TeachFirsters, compared to those in the same school who did not.

Is Teach First workingDEPT

The next set of feedback we received claimed that…

The effects are too large small

It seems most unlikely four [Teach First] teachers can raise the attainment of a school or department single handed by teaching their pupils better, not least as they simply do not teach enough pupils.

[blog comment]

Does ‘5% of a standard deviation’ represent significant and positive impact?

[@David_Cameron76 on twitter and in the audience at researched!]

This poster asked the far more important question…

The effects are too large small, given costs

Are these estimated marginal benefits worth the massive public subsidy that @TeachFirst receives? I don’t think so

[@jpjsavage on twitter]

We don’t answer this question in our research paper, but I hope we’ll be able to in a new Nuffield-funded project I’m working on which is led by Ellen Greaves at IFS. We’ve been asked to look at the relative costs and benefits of all the different teacher-training routes. There are two major impediments we face: no national database of school and departmental participation in PGCE programmes exists and SCHOOL DIRECT WAS NOT RANDOMISED (@samfr – this happened your watch did it not? Why no randomisation to stagger roll-out?).

Some commenters on our research did not believe our findings because…

Your research is clearly biased

…Clearly you have identified what you see as a benefit of this approach (albeit based on the work of someone who clearly has such a positive view of what Teach First are doing that he has gone to work for them)…

[Jonathan Savage as blog comment]

“You’ve seen improvements in these schools and have just gone looking for explanations…”

[seminar audience member]

I have no interest in Teach First in the sense that I wasn’t paid by them to do the research, I trained as a teachers via a traditional PGCE and the wider ‘narrative’ of the research I do isn’t underpinned by a theoretical view that programmes such as Teach First are likely to be successful. My co-author was trained by Teach First, worked in a secondary school whilst he conducted this analysis, and now works for Teach First. If people believe our research is biased it is hard for us to persuade them otherwise. It isn’t enough to claim that scientific and quantitative methods are less susceptible to bias, but this isn’t true (see this article on how doing maths is influenced by political beliefs). We should have been required to declare our methods – our outcomes variables and regressions we planned to run – before we conducted the research, as EEF evaluators are required to, but we didn’t.

We also had the usual comments from people who ask…

Can we ever ‘know’ anything?

You identify so many potential caveats, influencing factors, limitations to the validity of the methodology and other potential problems it is hard to take your findings seriously.

[Jonathan Savage as blog comment]

Paper ‘estimates’ benefits. Others claim them as facts.

[‏@jpjsavage on twitter]

Can you really claim direct causal links? Your paper is rightly tentative the headlines are not.

[@egwilson on twitter]

There are philosophical questions about what knowledge and causality are; I’m not the right person to blog on these things. But my problem with the commenters above is that by invoking the ‘it is not causality’ argument, they are taking a value position that there is still no evidence that Teach First works. For me, research on policies should simply aim to shift the balance of probabilities on whether they work or not. Does our research alter the balance of probabilities that Teach First works? I think it does, though we can argue about how much. Am I convinced enough of my findings that I’d be happy to suggest to a headteacher that they join the programme? I think I would (though if they were really reluctant I definitely wouldn’t push it!).

Some commenters rightly ask what the ‘thing’ is that we’ve identified, or…

How should we interpret the effect?

Could you account for the vacancy-filling nature of the programme? That is, a mathematics teacher is better than no mathematics teacher?

[comment by email]

(My response: This is just part of the Teach First effect so we don’t want to ‘account’ for it in our estimation!)

I don’t think you can compare to other ITE methods, as not part of study?

[@BeckyFrancis75 on twitter]

(My response: Correct)

What if the ’cause’ is involvement in “ITE” for a school/department & not “TF“?

[@JohnClarke1960 on twitter]

(My response: Great question! And we’ll be able to say more about this once we’ve completed our Nuffield project)

Have you proved that Teach First impacts on poor kids within these schools, as opposed to other kids?

[a question at researched2013]

(My response: Great question! Let me crunch the data and get back to you)

Now we start getting onto the really important questions, as raised by @samfr in a blogpost…

How might Teach First plausibly change the pupil experience?

Is Teach First workingHOW

Is Teach First workingHOW2

And we had a few more great questions from the audience at researched2013:

Have PGCE courses seen a deterioration in applicants due to Teach First?
Why is Teach First more appealing than a PGCE and what role does the salary versus student loans play in this?
How can we know if Teach First or PGCE applicants have the qualitatively superior characteristics?
Is a major advantage of Teach First the two year length of the programme, compared to just one year for a PGCE?

Novices and Veterans: What new data tells us about teacher turnover and school deprivation

CMPO Viewpoint

Rebecca Allen and Simon Burgess

A new school year has just started, new students have just arrived – what about new teachers? Are there a lot of new faces in the staffrooms? One of the stories frequently told about schools serving poor communities is that they suffer from very high and damaging staff turnover. Few teachers stay a long time, and, relative to schools in the affluent suburbs, there is a constant ‘churn’ of staff. This lack of experienced teachers reduces the chances of new teachers learning the trade on the job, and means that both students and school leaders are forever coping with new names, personalities and teaching styles.

Is this true or urban myth? For the first time, we can start to answer this question systematically, moving beyond a collection of local anecdotes. New data collected from all schools about their workforce has the potential to hugely improve…

View original post 943 more words

How can London schools be so good, given the high cost of living for teachers?


Rebecca Allen

Chris Cook, the Financial Times education correspondent, has been writing about the Department for Education’s suggestion that the School Teachers’ Review Body (STRB)should consider whether greater variation in teachers’ regional pay is needed. He notes that greater variation in teacher pay would create a bizarre situation where schools in our most successful region (London) become even more generously funded, with a deterioration in funding in places where schools appear to struggle.

This observation raises the interesting question as to why London schools do so well, given that the high cost of living should make it difficult for them to recruit and retain the highest quality teachers. Why don’t the capital’s best teachers simply migrate to Stoke or Blackpool where their salary would provide them with a nice family home and a higher standard of living?

I would suggest that there are four possible explanations for this phenomenon, and it is…

View original post 537 more words

Reforming teacher training

CMPO Viewpoint

Rebecca Allen and Simon Burgess

This week the House of Commons Education Select Committee published its report on the teaching profession. This post gives the main points of our evidence to the Committee.

We think of Initial Teaching Training (ITT) as encompassing both the initial training and the probationary year. How should this be set up to produce the most effective teachers who will have the greatest impact on pupil progress? ITT plays two roles for the profession – training and selection with the emphasis typically placed on the former. Both are important and neither should be neglected, but we argue that the evidence suggests that if anything, selection is the more important, and this is our focus here. An important role for selection is completely standard for any professional accreditation system in either public or private sectors.

The key argument is this: the sharpest selection should be made…

View original post 942 more words

Where do star teachers come from? (via CMPO Viewpoint)

Rebecca Allen and Simon Burgess   This Sunday sees the culmination of the National Teachers Awards weekend, with a televised presentation of prizes. This seems very appropriate – in terms of the impact on learning outcomes, hardly anything matters as much as having a good teacher. This is not an empty platitude – research shows that the effect size of having effective versus ineffective teachers is very large relative to most educational int … Read More

via CMPO Viewpoint