Why use quizzes and timed exams in learning design?
We’ve all experienced the ups and downs of testing in formal schooling: cramming the night before to get ready… the horrible feeling when scoring badly on a final exam… and the sense of pride and accomplishment in doing well on a tough test.
As much as we hated tests, the fact is that testing is critical to structuring learning for maximum effectiveness. To increase knowledge and information retention you need to:
- Break the information into bite-sized chunks
- Spread the learning out over time, and
- Periodically practice recalling the information being learned.
That last one is where testing comes in. As mentioned in our article, “3 Ways for How to Structure Course for Knowledge Retention,” testing is critical for retrieval practice.
Quizzes versus Timed Exams
Quizzes test a learner’s understanding of the material that they are learning. They also provide a wake-up call, telling the learner that they truly are expected to learn the material and which content is the most important.
Timed Exams do the same, with the added functionality of placing a time limit on how long learners have to complete the exam once they begin. This creates a sense of urgency and importance, making the exam feel more formal and perhaps more official.
The challenge: creating GOOD quizzes and timed exams
A good quiz or timed exam leaves the learner pleased about their correct answers and understanding why their incorrect answers were wrong. A poorly written quiz or exam leaves them frustrated. Maybe the questions were unclear, maybe they feel they were tricked into giving a wrong answer, or maybe the quiz was too easy and therefore a waste of time. Depending on how upset the learner gets, they might shut down and resist using the information they’ve learned, or they might tune out completely and stop paying full attention to the rest of the training.
It’s not easy to write good quizzes and exams. Luckily, there are a few simple rules of thumb that can make a big difference.
Step 1: 12 Fundamentals for Writing Good Quiz Questions
When creating your quiz or timed exam, start by applying some basic fundamentals for writing questions well:
- Align with learning objectives.
The questions should focus on the most important skills and information being taught, often summarized by the module’s learning objectives. - Test usable knowledge and application.
When you’re writing a quiz, always ask yourself whether or not your questions are testing real knowledge and application. Focus on what you want them to be able to do with the information, as opposed to just testing if they have it memorized. Do you really want them to be able to list the 19 steps of a process? Or, when in a certain situation, do you want them to know what to do next or where to look up the information?
Take a look at the examples below. Which question tests actionable knowledge? Why? What’s different about the two?
- Keep it simple. Don’t bury the question in complicated language or unnecessary phrases. Instead of “Which of the following gizmos are used to…” ask “What is used to…?”
- Keep it positive.
Ask mostly positive questions: ask what they should do, not what they shouldn’t do. You want to reinforce what they should do, right? With “shouldn’t do” questions, perhaps ask about the consequences of doing things that they shouldn’t. - Avoid double negatives
Having negatives in the question stem and in the answer choices can result in learners choosing the wrong answer when they actually understand the concept. In this example, learners really have to think about the first answer choice here:
If you feel there is no other way to properly test understanding other than to use a negative, make sure you stress the negative to make sure learners don’t miss it. (“What is NOT an example of…”) - Be careful using absolutes.
Avoid using words such as “Always” and “Never” unless the answer really is such an extreme. Learners know that these absolutes are rarely true, so they are usually throw-away choices that make it easier to guess the correct answer. “Often” and “Rarely” will often work, though! - Try to avoid “All of the above” and “None of the above.”
In general, “All…” and “None of the above” answer choices make it easier to guess the correct answer. As soon as more than one answer looks true, learners know that “All…” is the correct answer. If you want to know that they know all of the items involved in a concept, change it to a multiple response question, throw in a reasonably possible distractor, and ask them to select all of the items that apply. - Aim for answer choices of a similar length.
With multiple choice questions, keep all of the answer choices about the same length. The answer that is much longer or shorter than the others is usually the right one and an easy guess for people who don’t actually know the answer. - Watch your grammar!
Answer choices should match the question and each other grammatically. For example, all of the choices should be present tense verbs, or all of them should be nouns. Likewise, if the question asks “Who…”, the answer choices should be nouns like the names of people. And make all answer choices singular or all of them plural, or word the question so either could be right. (“Which gizmo(s) is/are…”)
In this example, the one plural answer is obviously correct:
Try this instead:
- Use True/False, Yes/No, or questions with only 2 or 3 choices sparingly.
When writing a question with only 2 or 3 choices you risk making the correct answer too obvious. Carefully consider whether the question is challenging enough to test real knowledge. Ideally, all multiple choice questions will have the same number of choices, such as 4 or 5; multiple choice multiple response questions may have a few more. And all of the choices should be things that could possibly be the right answer. This means no joke answer choices. We want to test their understanding, and a throw-away answer choice makes it that much easier to just guess the right answer. - Always have at least one reasonable distractor.
If you have a multiple response question where learners need to choose all of the options that are correct, include at least one incorrect answer choice. It is a best practice and a standard format to have at least one wrong choice, so having all of the choices be correct is essentially a trick question. Which brings us to our next tip... - No trick questions or unclear questions!
Trick questions aren't funny, and learners get frustrated with courses and instructors who fool them into getting a wrong answer. Make sure the question is clear; try and think of all of the ways someone could misunderstand what you’re asking, and then clarify the question to avoid confusion.
It’s fairly easy to create poorly-written quiz questions. Being conscious of some of the most common mistakes is the first step in creating well-written ones!
Step 2: Tips for Quizzes & Timed Exams in NovoEd
Next, let’s look at some tips for putting the quiz or timed exam into the NovoEd platform.
- Consider if it is critical to tell the learners that their answer is right or wrong.
Short answer or essay questions will test the learner’s knowledge better than multiple choice questions, but they are subjective and can’t be graded automatically. The platform can mark responses right or wrong automatically if they are multiple choice, multiple choice multiple answer, number, or table questions. - Remember to mark which answer choice is correct!
This may be obvious, but it’s easy to forget to mark which answer choice is the correct one. Get in the habit of taking one last look to make sure all questions have the correct answer choice marked. - Add feedback that explains why each answer is right or wrong.
Use the Edit link to the right of your answer choices to add comments like, "That's correct" and an explanation of why this is the best answer or "Try again..." with feedback that explains why this answer is incorrect or that guides the learner towards the correct answer (without just giving it away). - Always include at least one question the learners must answer.
This one may also be obvious, but every once in while someone creates a quiz with a Read-Only statement in it and no questions. Learners will still have a Submit button which they can click, but they will not be marked as having completed the quiz even if they click that button. - Define rating scales when you use them.
When using a Likert scale question or any rating scale, define the scale clearly in the question stem. For example, “(1 = Strongly Disagree and 5 = Strongly Agree).” - Let your learners know the accepted format for responding to Number Questions.
When setting up the question, the answer must be a number in a certain format:- No commas to indicate thousands or millions, etc. (For example, "1000")
- No dollar signs or any other currency symbols
- No parentheses for negative numbers
- Instead, use a minus sign for negative numbers (For example, "-10")
- For the correct answer, decimals are allowed through 4 digits
- Correct answers with more than 4 decimals will be rounded to 4 digits (For example, "5.123456789" becomes "5.1235")
- RECOMMENDED:
Tell your learners the format to use. For example:
"SHOW ANSWER USING THE FORMAT: XXXXXX or -XXXXXX"
- Consider what will happen based on whether correct or incorrect.
What happens after each learner submits the quiz or timed exam? Besides the feedback listed in Tip #3, there are a number of settings that allow you to tailor the experience:- Quiz settings: If nothing happens when the learner submits their answers, should the quiz be for “Practice” instead of for “Assessment?”
- Quiz settings: If it is for “Assessment,” how many times can each learner take it?
- Timed Exam settings: When do you want learners to see their scores and feedback?
- Set this for a future date and time if you want everyone to take the exam before anyone sees their results. (And be sure to set this expectation in the instructions!)
- If you want them to get their results immediately, schedule the scores and feedback to be released at the same time the exam page goes live.
-
- Points: Do they earn a certain number of points for each correct response? Do they earn points for getting a certain percentage of the questions correct? (A passing score of 70% - 80% is common.)
- Completion: Does the quiz or exam satisfy one of the completion criteria for the course?
- Prerequisites: Does the quiz or exam satisfy the prerequisite condition needed to unlock another page of the course?
- Provide clear instructions.
Do they need to get a certain percentage of questions correct to pass? (70% to 80% is common.) Do they earn points for every correct answer? Do they earn a certain number of points as long as they get a certain percentage of answers correct? Is this an “open book” assessment? Can they refer to their notes? Can they search the internet for answers?
Use “Text only questions” to introduce the quiz or exam and state the instructions and expectations. You can even use them to delineate different sections.
With a timed exam, you may even want to have a separate page in the course with the exam instructions. This way learners won’t use precious exam time reading the instructions. Include information such as how much time they have and what happens if they run out of time. For example:
“The exam will become available on the next page of the course at 12:01am Eastern on August 1. It is due by August 8 at 11:59pm Eastern.
Once you begin the exam you will have 1 hour to complete it.
Please make sure you are truly ready to begin and will not be interrupted - you cannot pause the timer, and at the 1 hour mark the exam will lock and automatically save any answers you have already input.”
Wrapping It All Up
Thinking back, for the most part, our teachers didn’t quiz us to see us fail. They really were, for lack of a better phrase, testing us and using those tests to help us learn more. It’s how they tested us (and how well we did on those tests) that determined our reactions.
As Learning Experience Designers, our role is to think about the entire learning experience and deliberately create an environment that fosters learning. This includes using quizzes and exams in a way that complements and enhances the learning without alienating our learners.
Pop quiz! How are you going to use quizzes in the next piece of learning content you design?