Why might students fail in responding to an assessment task?
Cues, memory and cognitive load
A student’s cognitive response to an assessment task is not straight forward, as we discussed in this post. If a student provides a good response in an assessment, it is difficult to know how they have arrived at their response. If they have provided a poor response, it is even more difficult to work out why!
The most intuitive reason for a poor response in an assessment is that the student doesn’t know the answer. However, as we are about to see, it is more complicated than that.
As an example, let's consider a Geography teacher who wants their students to understand how oxbow lakes are formed. The teacher has identified the specific knowledge required for this understanding and has created a test question that asks students to provide a full explanation of the process with diagrams to illustrate it. For students who perform well in this test, we may be quite confident that they ‘know’ how oxbow lakes are formed. Therefore, a task such as ‘Explain fully, with the use of diagrams, how oxbow lakes are formed’ seems effective. However, what are we able to infer from poor performance?
For instance, if a student draws a series of diagrams accurately, but is unable to explain the process of formation clearly, what should we conclude? The student might have memorised the diagrams but have no understanding of what they represent. Or the student might understand that the diagrams represent a process over time but not know why the river is changing shape in the way shown. Or the student might understand the process but cannot recall the terminology needed to provide a clear explanation.
We may conclude that making inferences about the state of a student's knowledge of a concept is generally more straightforward when their knowledge is complete and firmly established rather than when it is incomplete and uncertain. However, if we aim to understand why a student is not performing well, which we must do if we are to use the assessment for formative purposes, then partial or insecure knowledge of a concept becomes significant. It is essential to identify what tasks partial or insecure knowledge of a concept will allow a student to accomplish.
Reasons to do with the assessment task
In this post, we considered the role that cues play in activating a student’s cognitive processes. An incorrect or insufficient response in an assessment may be due to cue problems: a weak cue-target match, distracting cues, or the absence of discriminatory cues. Asking a different question or framing an assessment task in a different way can provoke a very different response. We have discussed cues in some detail, so let’s look at the other reasons that students might fail to respond well to an assessment task.
Reasons to do with memory
Knowledge that may truly be said to have been ‘learnt’ has three conditions: stability, structural integrity, and the ability for it to be retrieved for later use. A poor response to an assessment task may be due to the failure of any or a combination of these factors.
Stability is a measure of permanence and therefore relates to the idea of forgetting. There are two ways of thinking about forgetting that may be helpful.1 Firstly, we can think of forgetting as a process of decay or structural deterioration, like a bridge which has not been maintained. When faced with an assessment task, students with decayed memories may provide answers that are only partially or loosely appropriate.
Secondly, we can think of forgetting as decreasing precision and increasing ‘fuzziness’ of memory. Over time, our minds tend to retain general impressions better than precise detail. We might have a sense that the day out at the beach was enjoyable and involved a variety of activities, but we are unlikely to remember the details of what we ate, what was said, or how long we spent sunbathing. Unfortunately, learning at school tends to demand that we remember lots of precise details – the date of that event, the name of that process, the reason for that change. Where precise memories have faded, students may respond to assessments with fuzzy answers. They know the gist of the answer, but not the detail required.
A poor response to an assessment task may also be due to the structural integrity of the student’s mental model. We want students to connect knowledge together in certain ways; to have a valid ‘schema’.2 If these connections are not made it will affect the student’s ability to answer more open questions and provide broad and balanced answers.
Finally, it may be that a student knows the answer but is unable to retrieve it.3 Poor retrieval is perhaps the most frustrating reason for a weak assessment response as it is not possible to distinguish between a student simply now knowing enough as opposed to not being able to retrieve that knowledge on demand.
Despite the difficulties of making valid inferences about the reasons for a poor response to an assessment, it is possible to design assessment tasks to reduce this difficulty. We call these types of tasks ‘diagnostic’ because they help the assessor establish not only whether something is known, but what the state of that knowledge might be. Diagnostic tasks may:
Provide cues or scaffolding which increase the likelihood that respondents will retrieve relevant knowledge.
Test both precise and gist memories.
Expose the connections between items of knowledge.
Expose misconceptions.
It is worth noting that exam questions are designed to assess whether students know something and not to support inferences being made about the reasons for less than perfect performance. For this reason, it is not appropriate to rely solely on past exam questions to construct an internal assessment programme. They won’t tell the teacher everything they need to know to improve learning.
Reasons to do with cognitive load
Although we recommend that teachers think about the three stages of activation, search, and response formation in a linear fashion, in practice, neuroscientists believe that response formation begins less than a second after comprehending a question and these processes occur concurrently. At each stage, the nature of the process will depend on the state of the long-term memory: specifically, the stability, structural integrity, and retrieval strength of relevant knowledge will determine how well the cognitive process functions.
However, it also depends on the ability of the student’s working memory to manipulate the information brought to mind by the cues. The manipulation involved can be significant when the process is controlled, rather than automatic, meaning that the information must be consciously attended to in the working memory. Since each student’s working memory has limited capacity, their ability to respond successfully to a task will depend, in part, on whether they have the capacity to manipulate all the information brought to mind by the assessment task. The size of this demand is known as the ‘cognitive load’ of the task.4
In general, the cognitive load of an assessment task depends on the state of the student’s long-term memory, including the automaticity of recall for the student. A task that leads to an automatic response by one student will require effortful cognitive processes in another student, such as search, discrimination, and deduction. Therefore, we cannot think of tasks as involving known cognitive activities, such as ‘problem solving,’ with certainty, since the likelihood of the student thinking in a particular way will depend on their past encounters.
Experiencing cognitive overload gives us one more reason why students may fail to respond to an assessment task successfully. Cognitive overload is more likely to occur when a student’s working memory is smaller and when their long-term memory does not facilitate automatic recall.
In this post, we have considered some of the reasons students may not provide a satisfactory response to an assessment task. In summary, these are as follows:
The assumed cue-target match does not exist.
Question cues are not sufficiently discriminatory.
What they once knew has been partially or completely forgotten.
The knowledge is not appropriately connected as a valid schema.
Retrieval strength is weak.
Memory interference disrupts the selection of appropriate responses.
Response formation is poor.
Working memory is overloaded.
The above reasons relate to the process of cognition. Of course, there are other reasons students may provide a poor response, such as distraction or disinterest. We will consider environmental, attitudinal, and behavioural aspects of assessment at another time. For now, it is enough to acknowledge that making assertions about what students don’t know is not straightforward!
What might we do to mitigate the effects described in this post? First, we can take care in how we construct assessment items. Second, we can make assessments more diagnostic i.e. more effective at diagnosing the deficiency in a student’s knowledge. Third, we can use a wide variety of assessment items over time to test students’ knowledge from different angles.
If you want to explore the idea of memory, we recommend this post by Marc Smith: The problem with gist – The Emotional Learner
This EEF post is a good starter on retrieval practice: EEF blog: Why bother with retrieval? | EEF
An introduction to cognitive load: An-introduction-to-cognitive-load-theory-v2.pdf