Reimagine Learning

True/False Questions

True/false questions are really multiple choice questions with two answers: true and false. They tend to get used in low-stakes assessments, to generate class discussion, or because sometimes learning designers want to include questions that are easier to answer. However, they tend not to be used in high-stakes assessment, for these reasons:

Lucky guesses

With only two choices, true/false questions are vulnerable to guessing. Completely random guessing yields the correct answer half the time, and so many, many true/false questions would have to be administered before any useful information can be gleaned. Even then, the information is shaky. If you ask ten true/false questions and someone answers seven of them correctly, have you learned anything? Probably not much. Completely random guessing will on average yield five correct answers, and those who are lucky will sometimes get six or even seven correct answers. Getting seven out of ten correct could also mean that someone knew the answers to three questions and guessed on the rest. Without a large number of responses, true/false questions provide little information about the real skills of the learner.

Too subjective, or too easy

The statement in a true/false question is not compared to any other statement. It must be judged entirely by itself. For this reason, it must be absolutely true or absolutely false, with no exceptions. Writing these statements is harder than it sounds, because almost every statement you can make will have an exception somewhere. Is it absolutely true that mammals don’t lay eggs? Generally, yes, but what about the platypus? Is age a legally-defensible job requirement? Generally, no, but what about bartenders? The US has 50 states, right? That sounds right, but technically, Kentucky, Massachusetts, Pennsylvania, and Virginia are commonwealths, so do they count as states? It depends on what you mean by “state.”

These examples illustrate a key weakness of true/false questions: if they aren’t written very carefully, they are too subjective, and in those cases the people who get them wrong may know the content better than people who get them right. To avoid these problems, question writers write these questions very starkly, with no room for doubt. That avoids the ambiguity problem but can make them very easy to answer.

When are they useful?

Some statements really are absolutely true or absolute false. Two is the only even prime number. A bat is a mammal, not an insect. There are not 538 states in the US. True/false questions can be used to test knowledge of these and similar concepts, but for the most part, they won’t be a part of high-stakes assessments. Even so, they can still be used to spur discussion or otherwise provoke thinking. Consider using them in live environments to provide interaction and generate discussion. Just keep the stakes low, and give encouragement to people who find exceptions that no one else (including the person who wrote the question) thought of.

1 Response

1 Response