At-home short duration open-book MCQ or short answer questions

An example

Multiple choice questions (MCQs) are a standard and time-proven assessment type for many scientific and applied subjects, and feature highly in many professional examinations.

Online testing is typically run in a short time-limited format similar or identical to an on-campus exam. Can be administered in open- or closed-book mode. (Referring to information sources limits time for remaining questionss in time limited format).

Short answer questions (SAQs) and very short answer questions (VSAQs) avoid prompting answers. VSAQs may be automatically marked (searching for key words), with oversight. SAQs require more manual marking. 

Pros

Questions can be surprisingly ‘workplace realistic’ including images or video, data interpretation, and scenarios. ‘Best of multiple’ answering rather than True/False improves accuracy and validity of questionss. Short answers remove prompting at expense of complexity and/or marking time.

This approach can cover wide areas of knowledge and skills through having many questions in one exam.

Automatic marking partially makes up for academic demands of question and exam setting.

Very reliable, if large enough – the student would get a very similar mark if they were to take the test a second time. Much more reliable than essay questions, for example. Can generate statistics on exam performance to reassure outside bodies.

Cons

Requirement for an appropriate exam platform, question bank, technical support.

Need to have a large question bank to allow for ‘question leakage’ to the student body. It takes a lot of academic time to write, quality assure and maintain good questions, and new ones are always needed.

For quality assurance and improvement, require statistical (psychometric) analysis and understanding, and familiarisation and training of exam board members.

If run in open book format, not suitable for questions where the answer can be Googled, though doing that to many questions will increase time pressure.

Assuring closed-book format currently requires advanced (expensive) technology to lock down device operating systems to prevent accessing Internet or files.

Moderate susceptibility to misconduct (see below).

Requirements

Short time limitation only possible where technology permits. For example achieved by continuous updating online rather than a manual uploading event at the end.

Clear guidance to students on behaviour expected and if open-book, what type of resource is permitted and what not. (Conferring with other individuals is generally not.)

Online exams in this format are stressful. Technology and administrative support mechanisms are fallible. It is essential to stress-test processes and familiarise students and staff in a full-scale formative format before running a summative exam.

Resistance to academic misconduct

Moderate. There is scope for sharing answers electronically, or consulting prohibited resources, or getting help from an individual. Time limits may reduce those options, but cannot eradicate them.

Making it more robust

• Online proctoring? A resource-intensive and expensive option that may reduce risk, but does not eradicate it. May have uses in professional or other critical circumstances where alternative options are not considered adequate to test learning outcomes. More on Minimising Cheating.
• Shorter duration? It is usual to run these tests in a synchronous and time-limited manner. This reduces scope for some types of misconduct.
• Other technical measures? Some platforms include mechanisms to make conferring or external input more difficult, include randomising question order, or preventing returning to previous questions (very unpopular with candidates), or locking down the device on which the exam is taken to limit other functions.
• Include other forms of assessment? For example, live oral assessments that overlap the same learning outcomes. These can carry marks, but may also reveal inexplicable differences between knowledge or skills demonstrated in two types of test that may justify further investigation.