Fifteen school data leads convene, bringing together their Heads of Science, English, History, and Maths to create shared end-of-year assessments in these four subjects.

What’s the impetus behind this collaborative effort?

The performance of a subject department is often inferred by comparing the knowledge and skills of their students against a benchmark—usually other students. However, making such inferences is fraught with difficulties when assessments are confined to a single school.

For instance, one might be tempted to compare results across departments within a school, but this proves problematic. A score of 73% or a Grade B in English does not inherently equate to the same level of achievement in History.

Another approach is to use the same assessment over several years, comparing the current Year 8 cohort against their predecessors. This method is not without its pitfalls. Even if the demographic makeup of students remains stable, one must be cautious of the ‘sawtooth effect’: marks tend to dip when a new assessment is introduced, gradually rise as teachers become familiar with it, and then drop again when the cycle restarts with a new assessment.

Comparing the performance of different classes within a year group may appear to be a reasonable alternative. However, this only holds water if students have been randomly allocated to classes— a rare occurrence. More often than not, students are purposefully placed in classes for various reasons, be it to separate certain individuals or even to match them with specific teachers.

Given these challenges, schools seeking to gauge the progress of their year groups in a subject can gain valuable insights by participating in shared assessments across multiple schools.

And so, fifteen school data leads bring their Heads of Science, English, History, and Maths together to design some shared end-of-year assessments in these four subjects.

But what could possibly go wrong?

The Complexity of Curriculum Timing: A Tale of Fifteen Science Departments

The Heads of the Science Departments from fifteen schools gather with a sense of optimism. They’re all following the same curriculum programme, so the task of creating a shared end-of-year assessment should, in theory, be straightforward.

However, the first hiccup arises when they realise the schools are sequencing topics in different orders. By June, some schools will have recently studied the topic of ‘Earth and Space,’ making it fresh in the students’ minds. Others will have covered ‘Sound and Light’ instead. The issue here is the ‘recency effect,’ which is especially strong in subjects where topics don’t overlap enough to provide repeated reinforcement throughout the year.

One might argue that the recency effect could balance itself out—every school will have an advantage in the topic they’ve most recently taught. This sounds reasonable, but it makes the exact balance of topics and difficulty of questions within topics crucial. Can a centralised entity ever perfectly balance this?

Another issue comes to light: one school is notably behind in the curriculum due to having fewer science teaching hours in Year 8, followed by more in Year 9. Should these students be penalised with questions on topics they won’t cover until next year?

One could simply agree to drop the topic, however, this solution changes the assessment domain and could be considered unfair to the other schools. Worse, it could incentivise teachers to deliberately delay teaching this topic if they know these won’t be included in the assessment.

The science Heads of Department do succeed in writing a common assessment. However, the differing pace and sequence of curriculum delivery across these science departments pose significant challenges for creating a shared assessment. These conflicts are far from straightforward to resolve, even when guided by a shared understanding of what constitutes proficiency in science.

The Challenge of Multiple Texts: A Tale of Fifteen English Departments

The Heads of English Departments from fifteen schools gather, buoyed by a shared understanding of key concepts and beliefs about what constitutes ‘good’ writing. However, they soon encounter a complex issue that puts their unity to the test.

Across the fifteen schools, a diverse range of texts have been studied. While there’s some overlap, such as everyone having studied one of two Shakespeare texts, the variety still extends to up to ten different works.

One approach could be to create a multi-text optional paper in the style of GCSE exams. In this format, everyone would answer questions related to Shakespeare, but the other questions would vary based on the other texts studied. However, how can teachers feel confident they have written questions of equivalent difficulty for each text? And how can they moderate scripts written across different texts to ensure consistency in grading? Exam boards are able to do this because they have scripts from hundreds of schools and can cross-check their consistency of grading by referring to student SATs results. A small set of schools cannot replicate this process, and will likely face teacher marker bias issues, since teachers would know which texts their school had taught.

The English Heads of Department manage to write a shared assessment based on the two Shakespeare texts and decide not to write common assessments for the other parts of their curriculum.

The Challenge of Context-Dependent Understanding: A Tale of Fifteen History Departments

The history departments from the fifteen schools find themselves grappling with an issue of almost insurmountable complexity. The crux of the problem lies in the minimal overlap in historical periods studied across the schools in any particular year group. Even when there is some commonality of the period, the focus can still vary. Take the British Empire in Year 8 as an example: some schools emphasise the experiences of the colonised, while others focus on trade and economic development.

The departments consider an optional paper that would cover enough periods for all students to answer questions. However, the variable emphasis and timelines for each period make it difficult to agree on a marking approach to any individual question they try to write.

One of the fundamental challenges in history education is that understanding key ideas — be it power, war, or leadership — is deeply context-specific. The ease with which a student can grasp notions of power within the context of the Tudors, for instance, offers little insight into their ability to understand power in the context of the U.S. Civil Rights Movement.

Faced with these complexities, the departments decide to abandon the idea of a shared assessment — when a subject’s understanding of abstract concepts is inextricably tied to specific contexts, and those contexts vary widely across schools, crafting a common assessment becomes a nearly impossible task.

When Everything Aligns: A Tale of Fifteen Maths Departments

In stark contrast to their colleagues in history, English, and science, the maths departments from the fifteen schools find common ground almost immediately. The key to their success? They are following the same (purchased) curriculum, taught in the same order and at the same pace across all schools.

Maths is a subject that lends itself to a high degree of consensus by virtue of its hierarchical knowledge domain. The structure of the knowledge domain (in school maths, at least) allows the departments to quickly agree on what the assessment should contain, from the importance of various topics to the types of questions that should be included.

It is no coincidence that the subject where the Heads of Departments find it easiest to write a common assessment is also the subject where commercial companies are able to write common assessments!

Conclusion

The mantra that often guides educational assessment is that meaningful comparisons can only be made when we have:

• Standardised tests
• Of a standardised curriculum
• Sat in standardised conditions
• With standardised expectations of stakes
• Reported with standardised meaning

The experiences of the Heads of Departments in science, English, history, and maths serve as case studies in the complexities of crafting shared assessments. These narratives underscore one pivotal point: the absence of a standardised curriculum makes the creation of shared assessments exceedingly difficult.

Yet, it’s worth noting that the other elements of standardisation are often more manageable:

• Standardised Tests require an agreed approach to consistent marking.
• Standardised Conditions necessitate that schools agree on the timing, location, and conditions under which assessments are conducted.
• Standardised Expectations of Stakes call for a common understanding among students of the assessment’s importance, thereby motivating a uniform level of effort.
• Standardised Meaning demands that schools agree on how to convert marked scripts into grades or scores for valid comparison.

In summary, while the challenges of creating shared assessments are formidable, particularly when curricula are not aligned, they are not insurmountable in all contexts. The maths departments show us that when conditions are right—most notably, through a standardised curriculum—the process can indeed be straightforward.

By examining the contrasting experiences of different subject departments, we gain a nuanced understanding of what it takes to make shared assessments a viable tool for educational evaluation.