Question order
How to order questions and tasks so that students can demonstrate their knowledge
In previous posts, we have written about the issues to resolve when writing a substantive, set-piece assessment: what curriculum content should be in scope, how should the assessment sample from this domain, and what tasks and questions to include?
In this post we will discuss the final issue which is the order of the tasks and questions.
The most common approach is to arrange questions in increasing order of difficulty, with easier questions appearing at the beginning of the assessment and more challenging questions appearing towards the end. This approach can help build students' confidence as they progress through the assessment, and it also ensures the weaker students do not waste time on questions that are beyond their capabilities.
However, the increasing difficulty approach is not always optimal because we want to ensure tasks are ordered in a manner that helps students plan their response time effectively. For example, open-ended questions can be a nice way to start an assessment since they often allow students of all abilities to write something in response, whilst giving more able students a chance to demonstrate their capabilities (i.e. they can be great discriminators). However, their open-ended nature means there is a risk that some students spend too long on the question in relation to the number of marks available. If this is a teacher-written assessment, one solution is to suggest a fixed number of minutes on this type of task.
Other approaches to ordering questions and tasks might be appropriate in specific circumstances. Simply ordering items randomly is fine where there is not a steep gradient of difficulty. Grouping items by topic can also work well if you feel there are benefits to concentrating the students’ minds on one topic at a time, provided you feel sure that weaker students will not spend too much time on questions beyond their capabilities. Where a stimulus-response style used, such as case studies or data-response, grouping questions which relate to the stimulus material will obviously be required.
Differing the order of questions between students is also possible when using technology. This may help prevent students being tempted to look over the shoulder of the student next to them to see what answer they have given.
In summary, in deciding how to order assessment tasks, consider:
How to build confidence and encourage students to complete as much of the assessment as possible.
How to focus students’ attention.
How to reduce the chance of cheating.
How to support students in managing their time.