Improving existing MCQs

Post Thumbnail

By Theresa Suart & Eleni Katsoulas

Writing and editing test questions is an ongoing challenge for most instructors. Creating solid multiple choice questions (MCQs) that adequately address learning objectives can be a time-consuming endeavor.

Sometimes you may have existing questions that are pretty good, but not quite where you need them to be. Similar to a house reno versus new construction, sometimes it might be worth investing the time improve what you already have. How do you know which questions need attention and how can you rework them?

Previous exams are analyzed to determine which questions work well and which don’t. This can provide some guidance about questions that can be improved.

To select questions for an MCQ renovation, you can start with checking out the statistics from last year’s exams (available from your curricular coordinator or from Eleni).

Two statistics are useful indicators for selecting individual questions for tweaking, rewriting or other fixes: Item Difficulty and Discrimination Index.

Item difficulty is a check on if questions are too easy or too hard. This statistic measures the proportion of exam takers who answered the question correctly.

Discrimination index differentiates among text takers with high and low levels of knowledge based on their overall performance on the exam. (Did people who scored well on the exam get it right? Did people who scored poorly get it right?)

These two statistics are closely intertwined: If questions are too easy or too hard (see item difficulty), they won’t provide much discrimination amongst examinees.

If questions from previous years’ tests were deemed too easy or too hard, or had a low discrimination index, they’re ripe for a rewrite. Once you have a handful of questions to rewrite, where do you start? Recall that every MCQ has three parts and any of these could be changed:Exam

  • The stem (the set-up for the question)
  • The lead-in (the question or start of the sentence to be finished with the answer)
  • The options (correct answer and three plausible but incorrect distractors*)

The statistics can inform what changes could be necessary to improve the questions. For one-on-one help with this, feel free to contact Eleni, however, here are some general suggestions:

Ways to change the stem:

  • Can you change the clinical scenario in the stem to change the question but use the same distractors? (e.g. – a stem for a question that asks students what the most likely diagnosis is based on a patient presenting with confusion with the correct answer being dementia, can be then re-written to change the diagnosis to delirium)
  • Ensure the stem includes all information needed to answer the question.
  • Is there irrelevant information that needs to be removed?

Ways to change the lead-in:

  • Decide if the questions is to test recall, comprehension, or application.
  • Recall questions should be used sparingly for mid-terms and finals (but are the focus for RATs)
  • Verbs for comprehension questions include: predict, estimate, explain, indicate, distinguish. How can these be used with an MCQ? For example: “Select the best estimate of…” or “Identify the best explanation…”
  • You can use the same stem, but change the lead in (and then, of course, the answers) – so if you had a stem where you described a particular rash and asked students to arrive at the correct diagnosis, you can keep the stem, but change the lead-in to be about management (and then re-write your answers/distractors).

Ways to change one or more distractors:

  • Avoid grammatical cues such as a/an or singular/plural differences
  • Check that the answer and the distractors are homogeneous to each other: all should be diagnoses, tests or treatments, not a mix.
  • Make the distractors a similar length to the correct answer
  • Ensure the distractors are reasonably plausible, not wildly outrageous responses
  • Skip “none of the above” and “all of the above” as distractors

As you dig into question rewriting, remember the Education Team is available to assist. Feel free to get in touch.

Watch for MCQ Writing 2.0 later this spring.


* Yes, there could be more than three distractors, but not at Queen’s UGME. The Student Assessment Committee (SAC) policy limits MCQs to four options.

One Response to Improving existing MCQs

  1. Kanji Nakatsu says:

    This is a timely blog addressing a very relevant issue. I believe that examinations are critical teaching tools at our disposal and are especially influential on student behaviour at the university level. Accordingly, one of our aims should be to expose our students to regular examinations that test their ability to respond at levels beyond mere recall of factual information. If we employ such exams right from the beginning of the curriculum they have the potential to direct students with respect to what is expected of them and how they can learn the principles then apply them to real life situations. In order to allow for more probing questions, and questions that involve thinking through a problem, we may have to ask fewer questions that take longer to read thoroughly and answer. Questions like these can be asked in multiple choice format but they will take significantly longer for the faculty to craft; an hour or more per question is probably reasonable. If we create examination questions with these aims in mind, we might wish to take the opportunity to ask questions that cross single-session and discipline boundaries.

Leave a Reply

Post Timeline

Students striving to make a difference in our community
Published Mon, October 23, 2017

One of the attributes that our Admissions Committee works very hard to identify in applicants is a commitment to service. This has multiple dimensions, involving service to both individual patients and communities. It’s therefore always very gratifying to learn of efforts such as that described below in todays guest article provided by students Lauren Wilson, Katherine Rabicki and Melissa Lorenzo. … Continue reading

Mentorship isn’t rocket science – or is it?
Published Mon, October 16, 2017

One of the most consequential communications in modern history took the form a letter sent by Albert Einstein to American President Franklin Roosevelt on August 2, 1939. “Sir: Some recent work by E. Fermi and L. Szilard, which has been communicated to me in manuscript, leads me to expect that the element uranium may be turned into a new and … Continue reading

Meet Jenna Healey, the new Hannah Chair in the History of Medicine
Published Mon, October 9, 2017

The new Jason A. Hannah Chair in the History of Medicine knows most Queen’s medical students aren’t going to memorize historical dates and events as a matter of routine, and that’s perfectly okay. Dr. Jenna Healey notes that instead focusing on dry facts – that these days can readily be looked up — one excellent use of history is “to … Continue reading

Curriculum Committee Information – July 27, 2017
Published Mon, October 2, 2017

Faculty and staff interested in attending Curriculum Committee meetings should contact the Committee Secretary, Candace Miller (umecc@queensu.ca), for information relating to agenda items and meeting schedules. A meeting of the Curriculum Committee was held on July 27, 2017.  To review the topics discussed at this meeting, please click HERE to view the agenda. Faculty interested in reviewing the minutes of the July … Continue reading

From campus to community: the Loving Spoonful Service Learning Project
Published Mon, October 2, 2017

By Steven Bae and Lauren Wilson, MEDS 2019 “Let food be thy medicine, and medicine be thy food” – Hippocrates Food. It is a vital part of our existence, and is a focal point in many cultures. Over the course of one year, a person who eats three meals a day consumes 1092 meals. It plays such a large role … Continue reading