OUCOM Multiple Choice Exam Policy
To ensure that our students have experience with board-style questions that will help prepare them for board exams, the following CAC policy has been established. 70% of multiple choice items shall be in the board-style structural format. When possible, item writers are encouraged to construct questions that assess basic science principles as they relate to clinical practice.
Well-written MC test questions based upon block objectives are the key to valid and reliable assessment. You should strive to get as many colleagues as possible to help write and edit. One idea is to hold a mini-murder board on the questions you write. Arrange a group of colleagues to review your questions together for accuracy, clarity, grammar, adherence to guidelines, difficulty level and readability.
For multiple-choice items there are two basic formats:
v One-best-answer items involve a question or incomplete statement followed by five distractors, only one of which is correct. There are two types of these items:
Ø A single item is independent of all other items.
Ø A case is a group of one-best-answer items preceded by a header, which sets up a scenario to be used in completing the items that follow. Each item in a case should:
§ be dependent on the case history
§ not give away the answer to any other item
§ attempt to answer the question without looking at the answer choices
v A matching set includes a list of similar words or phrases (distractors) followed by numbered items (case scenarios) to be matched. The distractors should be alphabetized. You should not have the same number of items as distractors. Each distractor may be used once, more than once, or not at all. Matching sets are not one-to-one matches. The sets should involve some degree of interpretation and/or analysis.
v The stem should be in the form of a question and a complete sentence with enough information to be able to surmise the answer.
- STEM “LITMUS TEST:” You should be able to cover the answer & distracters and have enough information in the stem to surmise the answer.
v Make each distractor concise, equivalent in length and style to the others, and plausible. Each distractor should follow grammatically from the stem, with the majority of the information included in the stem, not in the distractors.
v Test important concepts, preferably using questions testing higher-order thinking. (See Bloom’s Cognitive Taxonomy and key words at http://www.nwlink.com/~donclark/hrd/bloom.html)
v Use only generic drug names.
v Do not be too general. There should be a specific objective/focus evident in the stem. The stem should lead to one specific answer. The item should be so focused that the examinee can determine the answer even before looking at the answer options.
v Do not teach in the stem. Avoid the use of unnecessary information
v Do not use negative phrasing. Do not use stems including the words EXCEPT, LEAST, or NOT.
v Do not use relative terms. Words like ALWAYS, NEVER, FREQUENTLY, and RARELY do not belong in any item.
v Each distractor must be independent. "All of the above", "none of the above", and "A and B are both correct" are not acceptable distractors.
v No True/False questions.
v The question setting should be specific to a particular patient.
Checklist for evaluating multiple-choice test items based upon the information above and “Classroom Applications of Educational Measurement,” 2nd edition, by Alber Oosterhof.
___ 1. One-Best-Answer items with a question or incomplete statement followed by five distractors, only one of with is correct
___ a. Does this item measure the specified skill?
___ b. Will all or most content experts agree on the correct answer to this item?
___ c. Is the language used in the item clear and at the student’s level?
___ d. Does the stem clearly present the problem to be addressed?
___ e. If the item is intended to measure a skill other than recall, does the stem present a question or problem situation that is novel to the student?
___ f. Is extraneous content excluded from the stem?
___ g. Are adjectives or adverbs emphasized when they reverse or significantly alter the meaning of a stem or option?
___h. Do the options avoid repetitive words?
___ i. Is the grammar in each option consistent with the stem?
___ j. Are all options parallel in type of content?
___ k. Does the item exclude options equivalent to “all of the above” and “none of the above”?
___ l. Unless another order is more logical, are options arranged alphabetically?
___ m. Is each distractor plausible?
___ n. Are all distractor of approximate equal length (tendency is for correct to be longer)?
___ 2. Case: a group of one-best-answer items preceded by a header, which sets up a scenario to be used in completing the items that follow.
For a Case, is each item:
___ a. dependent on the case history?
___ b. not give away the answer to any other item?
___ c. attempt to answer the question without looking at the answer choices.
___ 3. A Matching Set includes a list of similar words or phrases (distractors) followed by numbered items (case scenarios) to be matched.
___ a. Not that same number of items as distractors.
___ b. Each distractor may be used once, more than once, or not at all.
___ c. Matching sets are not one-to-one matches.
___ d. The sets should involve some degree of interpretation and/or analysis.
Other MCQ resources:
A Multiple Choice Test Writing handbook is designed to (http://web.uct.ac.za/projects/cbe/mcqman/mcqcont.html):
familiarize staff with the benefits and limitations of using Multiple Choice Questions (MCQs);
explain how the Technology Assisted Testing (TAT) programme at UCT can help individual lecturers and departments;