Using the IDEAL banks of questions for your assessments

Obtaining IDEAL Consortium Questions

Queen’s School of Medicine has joined the IDEAL Consortium, an international assessment item-sharing collaboration among Schools of Medicine. The Consortium has 27 member schools from 11 countries. Queen’s and UBC are currently the only Canadian members.

The IDEAL Restricted Question Bank contains over 20,625 assessment items including 17,109 MCQs, 539 short-answer questions and 461 OSCE stations.  Collectively, members contribute about 4,000 new questions to the restricted and non-restricted question banks annually.

Restricted Bank:  Please contact your Curricular Coordinator to request sets of restricted bank questions in your subject area or questions on particular topics. (Zdenka Ko for Year 1, Tara Hartman for Year 2, Jane Gordon for Clerkship Rotations and Candace Trott for “C” courses in clerkship.) Restricted bank questions need to be kept secure, so they can only be used on final examinations. A Word document containing the questions (as well as their answers and “item numbers”) will be couriered to you, or you can request that a secure MEdTech community be created for you to share restricted questions with other faculty members in your course.

To use restricted questions on final exams, simply provide your Curricular Coordinator with the item number of each question and the order in which you would like the questions to appear on the final exam. If you are sharing restricted questions via a secure MEdTech community, you can copy and paste your question selections into a Word document and upload it to the Curriculum Coordinator’s folder in the secure community. It is important that the IDEAL restricted bank questions not be emailed except in password-protected Word files. The restricted questions must not be viewed by students except during the writing of final exams.

You can specify edits to any of the IDEAL items – including OSCE stations. If you edit the items yourself, please highlight your edits so that your Curriculum Coordinator can transfer the edits to the local copy of the IDEAL bank.

The old LXR bank contained many duplicate and triplicate questions, so please let your Curriculum Coordinator know the origin of each exam question (IDEAL? LXR? Original? From a colleague?) We especially need to know, for copyright and item submission reasons, if any questions did not originate at Queen’s. Questions that did not originate at Queen’s will be marked, “Do not submit to IDEAL”, but can be stored in the local copy of the IDEAL bank and used on Queen’s exams.

Unrestricted Bank:  Unrestricted bank items can be used in online quizzes, in clicker sessions, on midterms etc. Students can have full access to all unrestricted bank questions. Currently the MEdTech team is creating an interface for the unrestricted bank so that faculty members will have full access to the questions. At present, requests for emailed sets of unrestricted bank question sets can be sent to Catherine Isaacs (isaacs@queensu.ca).

4 Responses to Using the IDEAL banks of questions for your assessments

  1. Lewis Tomalty says:

    I just tested the system in starting to prepare the Mechanisms of Disease final exam. I have a few comments…..access to IDEAL is a great idea and I know it costs (a lot) but it sure isn’t a miracle cure for writing MCQ’s. Yes there are lots of questions but there are also many many duplicates that come in random order. A lot of the questions are also of very poor quality. However, they do give you ideas on topics so with a bit of editing you can come up with new questions.

    Second comment is that the search process and security are major hindrances. I would suggest that sending word copies by courier present more of a security risk than a secure electronic search process. Paper copies get lost a lot easier than electronic. In addition, the results of the search just gives the teacher hundreds of questions in a Word document that then requires hours of wading through to get the questions you want. I sincerely believe we need a much more steamlined process (including the search process) or the bank will not be used to the degree we had all hoped for when signing up.

    • Hi Lewis: thanks for your feedback! Because the IDEAL databank is a part of Sue Chamberlain’s and Catherine Isaacs’s work, I’ve asked them for their responses to your points. So the following is from them:

      We have had mixed reviews regarding question quality. Some faculty members have been very pleased and others disappointed. As you have pointed out, even poor quality questions can give faculty members ideas for question topics.

      One innovation we’re working on is a web interface through MedTech for the unrestricted bank which will allow faculty members to search for and retrieve unrestricted questions themselves. We agree that supplying restricted question sets via a secure MEdTech site is preferable to sending them on a CD via courier. We’ll send out a note when this interface is up and running.

      Unfortunately because of our agreement with IDEAL, only the Coordinators can be given security to the server where the restricted bank resides, so only they can create question sets.

      Re. duplicates:
      If duplicates are observed within the same question set, please let Catherine know the item numbers so that we can inform IDEAL.

      If a faculty member has ordered one search by an IDEAL category and another by a related keyword, it is likely that almost all of the questions listed in the keyword search question set will also appear in the category search question set.

      Re Word Searches:
      Unfortunately true Boolean searches (“and”, “or”, “not”) cannot be conducted in IDEAL. We recommend that simple Word searches be conducted in the documents (top right hand of page: “Search in document”. It may also be helpful to use as much focus or specific concepts as possible to identify questions that may better match particular objectives. (Or that can be edited to match objectives – please feel free to edit any IDEAL questions. Simply let the Curriculum Coordinator know that the wording has been changed so that the edits can be transferred to the local copy of IDEAL).

      Hope these help.

  2. Catherine Lowe says:

    I echo Dr. Tomalty’s comments re the IDEAL bank completely. My experience was exactly the same when I tried to use the bank for the GI/surgery course. Like him I managed to get some ideas for new MCQ’s but not nearly as helpful as I would have wished.

    • Hi Cathy: Thanks for your comments. Catherine and Sue are working on a way to help with one issue; see below response to Lewis. As well, Catherine just wrote to say that after terms 2b/4b we will be conducting a survey to determine the overall level of satisfaction.

Leave a Reply to Catherine Lowe Cancel reply

Post Timeline

Curriculum Committee Information – September 28, 2017
Published Wed, November 15, 2017

Faculty and staff interested in attending Curriculum Committee meetings should contact the Committee Secretary, Candace Miller (umecc@queensu.ca), for information relating to agenda items and meeting schedules. A meeting of the Curriculum Committee was held on September 28, 2017.  To review the topics discussed at this meeting, please click HERE to view the agenda. Faculty interested in reviewing the minutes of the September … Continue reading

Grade Inflation – the “dirty little secret” of academia
Published Mon, November 13, 2017

“Would any of us have gotten into medical school today?” This was the tongue-in-cheek question I posed to my classmates at our medical school reunion last year. They were rather amused by it and, being very much aware of the high academic standards required by our current admissions processes, believed the answer was an obvious “no”. I tried to raise … Continue reading

Facebook thinks I’m a doctor…
Published Mon, November 6, 2017

  And other unusual things that happen when you’re an educational developer at a medical school It’s a unique and interesting thing being one of the non-medically-trained employees who work (mostly behind the scenes) to help run the undergraduate medical education program at Queen’s. On the one hand, friends and family can sometimes think I’ve magically completed medical school in … Continue reading

Nominations open for next Exceptional Healer Award
Published Mon, October 30, 2017

Instilling the values of patient-centered care is one of our goals in the UGME program. It’s also what the Kingston Health Sciences Centre Exceptional Healer Award recognizes in physicians from both the Hotel Dieu and KGH sites. Launched earlier this year, the Exceptional Healer Award is sponsored by the KHSC Patient & Family Advisory Council. It honours a physician who … Continue reading

Students striving to make a difference in our community
Published Mon, October 23, 2017

One of the attributes that our Admissions Committee works very hard to identify in applicants is a commitment to service. This has multiple dimensions, involving service to both individual patients and communities. It’s therefore always very gratifying to learn of efforts such as that described below in todays guest article provided by students Lauren Wilson, Katherine Rabicki and Melissa Lorenzo. … Continue reading