This post appeared in the now-defunctCentral Texas Instructional Design blog on this date.
Distracters are opportunities to choose incorrectly on a multiple choice assessment. The more distracters a question has, the less likely a correct answer results from a lucky guess. I mentioned last time that most of the companies I work with have standardized on a using four options (three distracters and one correct answer). Assessments in higher education frequently use an extra distracter to reduce the chance of guessing (CERNet, n.d.).
So, if having more options makes guessing harder, why standardize on four options?
- Four options is the point of diminishing returns.
- Writing good distracters is difficult.
Most of us are familiar with the point of diminishing returns from Economics 101. For a certain amount of work, we derive a certain benefit. At some point, we hit the point where there is not enough additional benefit to justify the additional work. The following chart and table show the returns on the work of writing additional distracters.
| Number of Distracters | Chance of Guessing | Difference |
|---|---|---|
| 1 | 50.000% | |
| 2 | 33.333% | 16.667% |
| 3 | 25.000% | 8.333% |
| 4 | 20.000% | 5.000% |
| 5 | 16.667% | 3.333% |
| 6 | 14.286% | 2.381% |
| 7 | 12.500% | 1.786% |
You can see that learners have a 50% chance of guessing correctly on a True/False question or a multiple choice question with only two options and one correct answer. Adding a third option reduces their chances of guessing by 16.7%, rounded. Going from four to five may still be worth the additional effort. But by the time you get to six options, the gain in accuracy is probably not worth the effort.
So, what is the correct number of options? It depends on the question and what the options are. When I took the written assessment for my driver’s license, one of the questions asked about the meaning of that little stripped sign you sometimes see at the roadside. One of the distracters was, “Stop for roadside barber shop.” My guess is that that distracter was never chosen, and its only reason for existence was to meet the magical number of required distracters. Since this distracter does not really distract, this was essentially a three-option question.
To sum up, there is no magic number of distracters. Remember that distracters should be “seductive alternatives” (University of St. Thomas Academic Support). They should “compellingly and confusingly” attract the test taker (Randall, 2003). If there are only two plausible alternatives, don’t waste your time trying to come up with two more implausible distracters that probably won’t have any effect on the outcome of the assessment.
Coming soon: Seductive Distracters
References
- Croation Academic and Research Network. (n.d.). 10 golden rules for writing multiple choice questions. Retrieved May 15, 2008, from
Self-assessment and summative assessment in e-education . - Randall, V. (2003, May 17). Recognizing distracters and foils. Retrieved May 15, 2008, from Passing the Bar.
- University of St. Thomas Academic Support. (n.d.). Multiple choice strategies for psychology tests. Retrieved May 15, 2008.
No comments:
Post a Comment