Another great post both, thank you. I’m again reminded that these messages about the relative unreliability of assessment data should be known by governors and trustees, who tend to think that grades are absolute and unquestionable. As senior leaders, we should be reminding our boards of this instead of allowing them to continue with a misconception.
I’ve always felt comfortable with unreliability of assessment, as long as everyone around me understands it too. It’s where people don’t understand the impossibility of reliability (and validity) that things are tricky (hence comment above re governing and trust boards)
It’s been useful to me to consider what broad inferences I can take from assessment data, to make best bets on the next course of action.
For example, allocating interventions based on data (which is what a lot of trusts did with national tutoring funding post Covid) has always seemed a bit bonkers. I think it can guide the decision, but the data will mean that some kids getting the intervention won’t really need it, or won’t be interested in it - and some not getting it, would have benefitted massively from it. Better to make decisions based on inferences drawn from triangulating with soft data.. staff knowledge for example.
It’s the point you’re making above.. reliable enough ‘for what?’ Or perhaps, we should ask - is assessment data ‘unreliable’ enough to avoid or be cautious about certain assumed next steps? :)
The Ofqual research should be widely publicised - an English exam answer has only a 50% chance of receiving the ‘right’ grade. If teachers knew this we might teach proper writing rather than spurious exam skills.
As good as I hoped! The core of it for me is in allowing students to conceptualise the assessment as a snapshot of something, not the thing itself. I always used to bang on about grades being like apples, but education being the tree. There’s a pretty close relationship between the two, but it’s the tree you need to focus on.
Another great post both, thank you. I’m again reminded that these messages about the relative unreliability of assessment data should be known by governors and trustees, who tend to think that grades are absolute and unquestionable. As senior leaders, we should be reminding our boards of this instead of allowing them to continue with a misconception.
I’ve always felt comfortable with unreliability of assessment, as long as everyone around me understands it too. It’s where people don’t understand the impossibility of reliability (and validity) that things are tricky (hence comment above re governing and trust boards)
It’s been useful to me to consider what broad inferences I can take from assessment data, to make best bets on the next course of action.
For example, allocating interventions based on data (which is what a lot of trusts did with national tutoring funding post Covid) has always seemed a bit bonkers. I think it can guide the decision, but the data will mean that some kids getting the intervention won’t really need it, or won’t be interested in it - and some not getting it, would have benefitted massively from it. Better to make decisions based on inferences drawn from triangulating with soft data.. staff knowledge for example.
It’s the point you’re making above.. reliable enough ‘for what?’ Or perhaps, we should ask - is assessment data ‘unreliable’ enough to avoid or be cautious about certain assumed next steps? :)
Thought provoking and robust as ever. Thanks.
The Ofqual research should be widely publicised - an English exam answer has only a 50% chance of receiving the ‘right’ grade. If teachers knew this we might teach proper writing rather than spurious exam skills.
As good as I hoped! The core of it for me is in allowing students to conceptualise the assessment as a snapshot of something, not the thing itself. I always used to bang on about grades being like apples, but education being the tree. There’s a pretty close relationship between the two, but it’s the tree you need to focus on.