Attention to take-home exams has increased rapidly over the last few months, since all students are at home and the only possibility to assess them is at home (or at least not on campus or in-class). To consider if take-home or open book exams can replace your original exam, we need to know what these exams are exactly, and what is aimed with these exams. In this article Marlies van Beek provides common definitions from recent literature on assessment in Higher Education, as well as some considerations and practical guidelines when designing these exams.
Differences between take-home and open book exams
At Utrecht University take-home exams and open book exams are sometimes used interchangeably. However, there are some specific differences that seem relevant when choosing one of these assessment methods.
Take-home exam
An exam students can take at any location of their choice. It can be send digitally to students, and students are asked to send it back to their teacher, or upload it, within a certain time limit. It is non-proctored, and the time limit is usually extended to days (rather than hours). During the time of the exam, students are allowed to use tools and resources and online information. Take-home exams are defined always as ‘high stake’, i.e., they have a decisive impact on the students’ grade. This as opposed to ‘home assignments’ which usually are ‘low stake’ (and cover less material).
Open book exam
An exam that is designed in a way that allows students to either refer to their class notes and summaries, or textbooks, or other approved material while answering questions. Usually the time is limited, i.e., 2 hours, and it is usually a proctored (invigilated), in-class exam.
Take-home exam | Open book exam | |
Duration | extended to days | limited: i.e. 2 hours |
Allowance of use of material | usually completely unrestricted | can be restricted to approved material only, or unrestricted and students are free to bring whatever they want |
Proctoring | non-proctored | proctored in-class |
Considerations for redesigning your exam
A great advantage of take-home exams often mentioned is that they reduce students’ anxiety (less time pressure) and they are an excellent tool when it comes to assessing students’ higher-order thinking skills (Bengtsson, 2019). The latter brings us to the main question when considering an alternative for your exam: what are the learning objectives of your course. This is a matter of validity; what do you aim to assess, and which assessment method is appropriate for that aim. As you can imagine, some learning objectives are more appropriate for a take-home exam than others. In literature we find that generally speaking, the higher the taxonomic level of the learning objectives, the more appropriate to assess those objectives by a take-home exam (Bengtsson, 2019). By assessing knowledge, Bloom’s taxonomy is often used. A take-home exam can assess the higher-order thinking skills, such as apply, evaluate, analyse. These higher-order skills require more time on thinking and reflection.
Another consideration might be the authenticity: which performance is closest to the performances required in the professional field? For instance does the student need to take time, use his/her access to online material etc. to perform well (like a take-home exam), or does the student need to be able to answer questions immediately (like an exam in-class)?
Redesigning the exam to a take-home exam, sometimes also means redesigning the online classes
Last but not least is constructive alignment: if the learning objectives are on a higher-order level, you might want to opt for a take-home exam. Redesigning the exam to a take-home exam, sometimes (if not already provided) also means redesigning the online classes, in order for it to be sufficiently aligned with each other: if the nature of the exam questions changes, strategies for preparing students to take those exams will also have to change. It will no longer be enough to paraphrase or simplify the content of the text books in the classroom. Teachers will have to design tasks that will provide exercises for the appropriate cognitive skills required in each subject. Instead of the teacher talking most of the time and students taking notes, classes will have discussions, questions, and other active processes. In other words, teaching will no longer be the transfer of information from the teacher to the student, it will be the training of the mind in certain cognitive skills (Mohanan, 2020). Apart from these teaching techniques, students also need to be prepared for the online assessment.
Since take-home exams are not proctored, the apparent risk of unethical student behaviour is mentioned often. This is a matter of reliability: is the chance of irregularities and fraud enhanced? Do students for example, share or discuss possible outcomes? Approaches to these questions vary. For more authentic exams, and/or when a personal reflection of the student is required, this issue might not be a problem at all: in many professional fields, finding a sparring partner to discuss possible outcomes before taking action is required or appreciated. By choosing the assessment method it is advised to answer this question beforehand: is it necessary to prevent students from coordinating outcomes. Research shows that for take-home exams with which higher-order learning objectives are assessed, this is often not the case. On problems or cases with multiple answers or multiple ways of solutions, coordinating answers is seen as added value.
For open book exams however, students are not allowed to work together or coordinate possible answers, so this should be prevented (that is why this is originally done in-class, proctored). In the current situation with online exams, to manage this reliability issue, various measures are taken. Bengtsson (2019) gives an overview of research on these measures. Examples of these measures, sometimes taken in combination, at Utrecht University are:
- ask for proof (i.e. notes or pictures) and justifications to all answers
- introduce an honor code
- submit electronically to permit plagiarism control (i.e. in BlackBoard)
- answers must make direct references to course-specific material
- make questions highly contextualized
- randomly scramble the order of questions
- implement remote invigilation services (such as Proctor exam)
- ask for hand-written answers
As already mentioned, the main premise for take-home or open book exams is that teachers can devise questions that require students to answer in more critical and analytical ways, thus encouraging higher-order thinking skills in their students; as compared to closed book or traditional exams that tend to encourage rote learning (memorization technique based on repetition) and more superficial application of knowledge.
This aim to assess the higher-order thinking skills seems to be underpinned by the students’ perspective as well. Results of the student evaluation on period 3 of the year 2019-2020 at Utrecht University, specifically evaluating the online education and assessment (Eindrapport Evaluatie ‘onderwijs op afstand’ door studenten, 09-06-2020), shows that also students mention the wish to be challenged on these levels: they show a preference for take-home exams. One of the students stated: ‘…take home exams take the emphasis off of memorization and should be implemented more generally even if not necessary. We're not in high school anymore, why are we prioritizing memorizing over understanding?’
Practical guidelines
Special attention needs to be given to the construction of take-home or open book exams.
- Make use of a blueprint (i.e. test matrix, table of specification) and keep the learning outcomes in mind i.e. what skills and knowledge are you assessing?
- Use the experience of professionals in the field by collecting material for the case
- Before taking the exam, present it to your colleagues or to a panel of experts
- Change the description of the case and/or the question if there is insufficient agreement on the correct outcome among colleagues
- Keep the scoring mechanism simple
- Try to co-work on case-based exams with other study programmes or institutes
- Try to integrate types of case-based questions, and how to answer them, in your learning activities during the course
- Provide students with some examples of questions (and answers) to increase transparency
- Use the representation of real situations; avoid presenting situations that do not exist/ are not plausible in real life
- Make sure that the description of the case is as clear and specific as possible
- Make sure there is sufficient information on the content and the context of the case
- Present the information or data uninterpreted; present relevant qualitative or quantitative data and then ask students interpretative and application questions – What does the data show? What relevance does this data or does the scenario have in terms of [component of current topic]? What other factors could potentially affect this data? How would you test for these?
- Be aware of irrelevant information (distraction)
- Questions need to be devised to assess the interpretation and application of knowledge, comprehension skills, and critical thinking skills (rather than only knowledge recall)
- Present relevant qualitative or quantitative data and then ask students interpretative and application questions – What does the data show? What relevance does this data or does the scenario have in terms of [component of current topic]? What other factors could potentially affect this data? How would you test for these?
- Relate the questions directly to the case
- Direct the question to essential decisions as much as possible
- Restrict the amount of decisions asked for per case
- Devise clear and unambiguous questions to limit student confusion and time spent interpreting the question
- Make sure the content of the question determines the form of the question and not vice versa
- Make sure that intended correct outcomes are completely correct and that incorrect answers are incorrect by all means
(From: Jaap Milius, O&T- UU, and van Berkel, Bax & Joosten-ten Brinke, 2014: Toetsen in het Hoger Onderwijs and a guide for academics- open book exams, Centre for Teaching and Learning, the University of Newcastle, Australia)
Contact
For requests on advice, please contact the author Marlies van Beek.
drs. Marlies van Beek
Email: m.e.vanbeek@uu.nl
Phone: 06 49 880 229
Further reading
For further reading on take-home exams and open book exams also see:
Bengtsson, L. (2019). Take-Home Exams in Higher Education: A Systematic Review. In: Educational Sciences
A guide for academics- open book exams, Centre for Teaching and Learning, the University of Newcastle, Australia
Mohanan, K.P. (2020). Open book examinations.
