Improving existing MCQs

Post Thumbnail

By Theresa Suart & Eleni Katsoulas

Writing and editing test questions is an ongoing challenge for most instructors. Creating solid multiple choice questions (MCQs) that adequately address learning objectives can be a time-consuming endeavor.

Sometimes you may have existing questions that are pretty good, but not quite where you need them to be. Similar to a house reno versus new construction, sometimes it might be worth investing the time improve what you already have. How do you know which questions need attention and how can you rework them?

Previous exams are analyzed to determine which questions work well and which don’t. This can provide some guidance about questions that can be improved.

To select questions for an MCQ renovation, you can start with checking out the statistics from last year’s exams (available from your curricular coordinator or from Eleni).

Two statistics are useful indicators for selecting individual questions for tweaking, rewriting or other fixes: Item Difficulty and Discrimination Index.

Item difficulty is a check on if questions are too easy or too hard. This statistic measures the proportion of exam takers who answered the question correctly.

Discrimination index differentiates among text takers with high and low levels of knowledge based on their overall performance on the exam. (Did people who scored well on the exam get it right? Did people who scored poorly get it right?)

These two statistics are closely intertwined: If questions are too easy or too hard (see item difficulty), they won’t provide much discrimination amongst examinees.

If questions from previous years’ tests were deemed too easy or too hard, or had a low discrimination index, they’re ripe for a rewrite. Once you have a handful of questions to rewrite, where do you start? Recall that every MCQ has three parts and any of these could be changed:Exam

  • The stem (the set-up for the question)
  • The lead-in (the question or start of the sentence to be finished with the answer)
  • The options (correct answer and three plausible but incorrect distractors*)

The statistics can inform what changes could be necessary to improve the questions. For one-on-one help with this, feel free to contact Eleni, however, here are some general suggestions:

Ways to change the stem:

  • Can you change the clinical scenario in the stem to change the question but use the same distractors? (e.g. – a stem for a question that asks students what the most likely diagnosis is based on a patient presenting with confusion with the correct answer being dementia, can be then re-written to change the diagnosis to delirium)
  • Ensure the stem includes all information needed to answer the question.
  • Is there irrelevant information that needs to be removed?

Ways to change the lead-in:

  • Decide if the questions is to test recall, comprehension, or application.
  • Recall questions should be used sparingly for mid-terms and finals (but are the focus for RATs)
  • Verbs for comprehension questions include: predict, estimate, explain, indicate, distinguish. How can these be used with an MCQ? For example: “Select the best estimate of…” or “Identify the best explanation…”
  • You can use the same stem, but change the lead in (and then, of course, the answers) – so if you had a stem where you described a particular rash and asked students to arrive at the correct diagnosis, you can keep the stem, but change the lead-in to be about management (and then re-write your answers/distractors).

Ways to change one or more distractors:

  • Avoid grammatical cues such as a/an or singular/plural differences
  • Check that the answer and the distractors are homogeneous to each other: all should be diagnoses, tests or treatments, not a mix.
  • Make the distractors a similar length to the correct answer
  • Ensure the distractors are reasonably plausible, not wildly outrageous responses
  • Skip “none of the above” and “all of the above” as distractors

As you dig into question rewriting, remember the Education Team is available to assist. Feel free to get in touch.

Watch for MCQ Writing 2.0 later this spring.


* Yes, there could be more than three distractors, but not at Queen’s UGME. The Student Assessment Committee (SAC) policy limits MCQs to four options.

One Response to Improving existing MCQs

  1. Kanji Nakatsu says:

    This is a timely blog addressing a very relevant issue. I believe that examinations are critical teaching tools at our disposal and are especially influential on student behaviour at the university level. Accordingly, one of our aims should be to expose our students to regular examinations that test their ability to respond at levels beyond mere recall of factual information. If we employ such exams right from the beginning of the curriculum they have the potential to direct students with respect to what is expected of them and how they can learn the principles then apply them to real life situations. In order to allow for more probing questions, and questions that involve thinking through a problem, we may have to ask fewer questions that take longer to read thoroughly and answer. Questions like these can be asked in multiple choice format but they will take significantly longer for the faculty to craft; an hour or more per question is probably reasonable. If we create examination questions with these aims in mind, we might wish to take the opportunity to ask questions that cross single-session and discipline boundaries.

Leave a Reply to Kanji Nakatsu Cancel reply

Post Timeline

Curriculum Committee Information – September 28, 2017
Published Wed, November 15, 2017

Faculty and staff interested in attending Curriculum Committee meetings should contact the Committee Secretary, Candace Miller (umecc@queensu.ca), for information relating to agenda items and meeting schedules. A meeting of the Curriculum Committee was held on September 28, 2017.  To review the topics discussed at this meeting, please click HERE to view the agenda. Faculty interested in reviewing the minutes of the September … Continue reading

Grade Inflation – the “dirty little secret” of academia
Published Mon, November 13, 2017

“Would any of us have gotten into medical school today?” This was the tongue-in-cheek question I posed to my classmates at our medical school reunion last year. They were rather amused by it and, being very much aware of the high academic standards required by our current admissions processes, believed the answer was an obvious “no”. I tried to raise … Continue reading

Facebook thinks I’m a doctor…
Published Mon, November 6, 2017

  And other unusual things that happen when you’re an educational developer at a medical school It’s a unique and interesting thing being one of the non-medically-trained employees who work (mostly behind the scenes) to help run the undergraduate medical education program at Queen’s. On the one hand, friends and family can sometimes think I’ve magically completed medical school in … Continue reading

Nominations open for next Exceptional Healer Award
Published Mon, October 30, 2017

Instilling the values of patient-centered care is one of our goals in the UGME program. It’s also what the Kingston Health Sciences Centre Exceptional Healer Award recognizes in physicians from both the Hotel Dieu and KGH sites. Launched earlier this year, the Exceptional Healer Award is sponsored by the KHSC Patient & Family Advisory Council. It honours a physician who … Continue reading

Students striving to make a difference in our community
Published Mon, October 23, 2017

One of the attributes that our Admissions Committee works very hard to identify in applicants is a commitment to service. This has multiple dimensions, involving service to both individual patients and communities. It’s therefore always very gratifying to learn of efforts such as that described below in todays guest article provided by students Lauren Wilson, Katherine Rabicki and Melissa Lorenzo. … Continue reading