Starting again with KS3 assessment

Q1. What is motivating me to change?

The desire to change is often motivated by a dissatisfaction with the status quo. When we talk to secondary school leaders, many express a frustration that their Key Stage 3 assessment isn’t right, but often they are not sure exactly what is wrong with it or what they should do about it.

For those readers outside the English education system, Key Stage 3 (KS3) is the first three years of secondary education (senior/high school) for children aged 11 to 14. It is wedged between primary school, the end of which is marked by national tests, and the beginning of GCSE qualifications which are studied for two years and sat at age 16. Since the end-of-key-stage national tests were scrapped in 2008, and national curriculum levels were abandoned in 2014, schools have been left to find their own way in assessing students.

Schools have invented, mimicked, and reinvented various KS3 assessment systems in the intervening years. Many take one of three forms:

1. GCSE Flight Paths: a data-led system whereby a benchmark GCSE grade is set for each student, in each subject, which, if achieved, would result in a neutral or positive value-added score. It is then assumed that students will progress in a linear way ‘along’ these flight paths. This logic requires that GCSE grades become the currency for assessment before GCSEs even begin to be studied. For example, assessments as far back as Year 7 will be graded 9 to 1, in line with the GCSE grading approach.

2. Banded grading: A variant on the flight path model uses bands rather than grades to communicate whether students are making the expected progress. These are often called things like Expected/Above Expected/Below Expected or Secure/Emerging/Developing. These bands can be defined in relation to age-related expectations or to some notion of what the performance expected of a typical student.

3. Test and rank: This approach uses tests at pre-determined points (at the end of year, at the end of curriculum modules, or at fixed points throughout the year) to rank students, often using a percentage or scaled score.

The above approaches are prototypical, but in reality schools often mash these up to create idiosyncratic systems, assessment philosophies, and lexicons.

It should also be noted that the above examples are all centralised and homogeneous approaches which seek, to various extents, to standardise practice across subjects and communicate assessment data in the same way. Decentralised approaches do exist, whereby subjects are asked to design their own approach which is meaningful to their subject. However, these are rare for two reasons. First, it is almost impossible for school leaders to collate data, make sense of it, and to feel any sense of control. Second, communicating to parents how their children are doing becomes challenging. The needs of these two stakeholder groups is often reason enough for schools to insist on a whole-school ‘system’, for better or worse.

Whichever approach (or hybrid approach) you use, you are likely to be dissatisfied in some way. This is because all of these approaches are flawed. It is also because there is no optimal solution to the problem. It is inevitable, therefore, that if you embark on change you will end up at some later point unhappy again. Given this, before you do anything, it is worth pinning down exactly what irks you and whether the costs of throwing everything up in the air are really worth it.

Let’s speculate as to what it is that might be bothering you.

We aren’t confident in the data

Assessment falls down when the data doesn’t tell you what it is you want to know. Whole-school assessment systems are often expected to answer the question of whether students are making the expected progress. But is this the right question to ask?

We wrote here about the difficulties of trying to measure progress. Progress means that students have learnt at least some of what you wanted them to learn - simple. But measuring progress isn’t simple. To begin, you must have a good idea of what students knew before. When a teacher begins a period of instruction, can they honestly say that they know what prior knowledge each student holds? If not, how will a test at the end of this topic tell us anything about the learning gains that have taken place? This dilemma means that what masquerades as progress in schools is often attainment in disguise. If you take the assessment data and map it across to a GCSE grade or descriptor, then use this to claim that the student has made the ‘expected progress’ (or not), you are creating a myth. At best, all you can say is ‘this is what the student knows now’. More probably, all you can claim is ‘this is how the student performed in this assessment’ because there will be a lot of noise in the data.

Furthermore, defining what students ‘should’ have learnt is problematic. Do you expect that students should have mastery of your carefully designed curriculum, or that they should have at least understood the basics, or that a particular assessment result (70%?) indicates sufficiency? It is even more difficult if you attempt to define what is satisfactory not only for the typical student but for each individual, given their past performance, aptitude, or ability in the subject.

To add one more dimension to this, how confident are we in the quality of subject-assessments which generate the data? If subject teachers do not have the expertise to design assessments which enable them to make reliable and valid inferences, or are not given the time to conduct these well, or are forced to assess in a certain way to fit the school’s approach, is the data telling us anything at all?

Schools tie themselves in knots over these questions. Faced with irresolvable dilemmas, abstract pseudo-measures of progress appear which create the impression of knowing - flight paths, banded descriptors, and statistical trickery.

Perhaps you are starting to suspect that your assessment system is creating a convenient illusion of certainty.

It doesn’t tell us anything we don’t already know

Ask any teacher who is doing well and who they are concerned about in their class and you will get a confident answer. Their knowledge will be built from a thousand imperfect impressions, from whether the student hands in their homework on time, to the quality of their work, to how well they did in a low-stakes test. When the ‘formal’ assessment takes place, there will be very few surprises.

So, we must ask the question: what does our assessment system tell us that we didn’t already know?

Of course, we can’t rely solely on impressions and spot-checks. Formal assessment performs important functions over and above other information sources:

(i) It mitigates bias. Bias occurs because teachers make unjustified assumptions about a student’s ability based on irrelevant data and internal prejudices such as how a student looks or how neat their handwriting is.

(ii) It isolates the signal. Formal assessment attempts to remove noise from the data by measuring performance in controlled conditions.

(iii) It enables attribution. Knowing how every student performed on the same task informs a judgement about an individual’s performance. For example, if every student performed poorly, the poor performance of a single student may be interpreted as a sign of weak teaching rather than individual failure.

Combined with the softer data teachers hold, formal assessments help us form a rounded, if flawed, picture of student learning. Unfortunately, school assessment systems often capture only the formal assessment data to make inferences about which students are and are not on track. This results in a situation whereby the data either confirms what teachers already know or is explained away as anomalous. When formal assessment data is fed into a spreadsheet and RAG rated against an expected trajectory or target, strange things happen. For example, teachers may find themselves in meetings with school leaders where they are presented with a list of ‘underperforming’ students which only vaguely reflects the teacher’s own notion of who needs additional support.

Perhaps you are starting to suspect that your assessment system is disconnected with reality. Have you stopped trying to improve real things and instead geared your actions towards improving data?

Subject leaders are just playing along

Assessment should look different in every subject. This is because subject disciplines have their own unique cadence, grammar, and progression model. We have written about this in some detail here, here and here. Disciplinary distinctiveness means that whole-school assessment systems force compromises on subjects which reduce the validity of inferences made from the data.

For example, if your system requires that all subjects sit a one-hour test at the end of the year, drama, PE, and music teachers will probably protest that this will tell them nothing meaningful about whether students are doing well in their subject. If you ask that all assessment outcomes are expressed as a percentage, this may makes sense for maths but not for Design Technology in which the assessment was to build a product. And if you standardise the timing of data collection to fit the cycle of parent reports, you will inevitably ask teachers to assess students at points which do not make sense to the way the curriculum has been structured.

When managerial convenience overrides disciplinary distinctiveness, teachers may give school leaders what they want and quietly get on with the task of meaningful assessment, the results of which are never shared beyond the department. This results in a two-tier assessment system and performative behaviours which keep school leaders and parents happy, whilst ignoring the truth that the system only makes sense for certain subjects.

If your assessment system has started to feel like a separate ‘thing’ to everyday practice - if teachers whisper out of earshot that what they are being asked to do just doesn’t make sense - perhaps there is some work to do in understanding the diversity of the curriculum.

Our assessment is a burden

The quest to measure learning can get out of control: workload spirals, invigilator and printing costs mount up, teachers spend more time marking and moderating than planning their lessons, learning time is diminished and lessons disrupted. And if the data generated through this gargantuan effort is not being used effectively to improve learning, you may be questioning whether it is worth all the effort.

Assessment systems can get out of control when there is a false sense of what is possible. We might believe that:

• a higher score always means more secure attainment, when in fact the assessment may have favoured one student over another.

• we can create a reliable profile of achievement across subjects, when in fact inter-subject comparability is very hard to achieve.

• we can identify gaps in knowledge, when in fact approaches like question-level analysis are unlikely to reliably indicate these gaps.

• poor performance is indicative of weak knowledge, when in fact there are many reasons for students responding poorly to an assessment.

• assessment are motivating desirable behaviours, when in fact over-testing can lead to gaming performance over securing learning.

Assessment systems can also get out of control when the ‘system’ is expected to do all the work. The formal whole-school system for assessment is only one aspect of the assessment ecosystem: a complex web of beliefs and actions that shape students’ experience of being assessed. Information resides outside of spreadsheets; behaviour is influenced by a million nudges; judgements are made through multiple, imperfect impressions. Harnessing assessment to deliver your goals takes more than designing an assessment system.

Are we any closer to isolating what exactly it is that your are dissatisfied about?

The source of your dissatisfaction relates to what you believe the purpose of KS3 assessment to be. You may view KS3 assessment as a stock taker - its purpose is to audit how much learning has taken place. Or you may believe the reason for all this effort is to inform intervention to correct for system failure. Perhaps you see the whole thing as an accountability method, enabling you to identify and act upon poor performance. Your goal may even be simply to be able to let parents know how their child is doing at school. Examining your dissatisfaction helps to reveal your assumptions about what assessment is for. This in turn will help you question your assumptions and set the goals for what a new approach should achieve, should you have the appetite to take on this challenge.

At 100% Assessment, we believe there is a higher goal which is that assessment should first and foremost promote learning. Are we satisfied that our current approach to assessment at Key Stage 3 is geared towards this purpose?

If your instinct is to start again with KS3 assessment, taking time to identify what is motivating you to change may mean you don’t make the same mistakes again.