June Curricular Leaders Retreat held: EPAs, Remediation and Feedback, oh my!
After bringing another busy semester to a close, UGME curricular leaders took time to reflect on the past year and take part in workshops and discussion groups on a number of areas of the curriculum at their semi-annual Curricular Leaders Retreat on June 19. The aim of the retreat was to share information and to generate ideas and solutions to address teaching and assessment challenges.
In his end of year report, Associate Dean Anthony Sanfilippo highlighted accomplishments of the past year as well as announced new faculty appointments.
After providing an entertaining and informative review of the process of curriculum renewal that UGME has undergone over the last several years, including the development of the “Red Book” objectives, Dr. Sanfilippo discussed how the emerging use of Entrustable Professional Activities (EPAs) will relate to and refine our existing curriculum and assessment processes.
Dr. Sue Moffatt presented an information session on how the three classroom-based “C” courses relate to both clerkship and the rest of the curriculum.
In a discussion about Service-Learning, led by Dr. Sanfilippo, faculty brainstormed ways additional service-learning opportunities could be created for medical students and others as well as ways they could support and encourage students in these endeavours. The Service Learning Advisory Panel will consider their suggestions and recommendations.
As a follow-up to last year’s popular workshop on remediation strategies, Michelle Gibson, Richard Van Wylick and Renee Fitzpatrick presented “Remediation 2” with additional cases and strategies.
For the afternoon, participants chose between a session on writing narrative feedback or one on making ExamSoft work for you.
Designed in particular for faculty working in clerkship, clinical skills and facilitated small group learning (FSGL), for the workshop on narrative feedback, Cherie Jones and Andrea Winthrop provided concrete examples and solutions to situations faculty routinely encounter when needing to provided constructive feedback to students. This included a discussion of ways in which oral and written feedback differ.
In the ExamSoft workshop, Michelle Gibson, Eleni Katsoulas and Amanda Consack worked with faculty to show how to tag mid-term and final assessments to match to assigned MCC presentations and Red Book objectives as well as coding for author and key word. Using these ExamSoft tools upfront makes it possible to use built-in reports to blueprint assessments, rather than having to do so manually. (For more on ExamSoft, check out the team’s poster from CCME at this link.)
To wrap up the day’s activities, pre-clerkship and clerkship course directors brainstormed with competency leads for ways the milestones identified for these intrinsic roles can be met throughout the curriculum. How to highlight and incorporate patient safety in different courses was also considered.
Documents from the Retreat are available to curricular leaders under “Retreats” on the Faculty Resources Community Page.
New career advisor appointed
Dr. Anthony Sanfilippo, Associate Dean of Undergraduate Medicine has announced that Susan Haley has joined the staff of the UGME Student Affairs office as a career advisor. She will be working with Kelly Howse and Renee Fitzpatrick in UGME’s growing Career Advising group.
An anesthesiologist, Dr. Haley has practiced in Kingston for 16 years. Prior to moving to Kingston, she worked at Mount Sinai at the University of Toronto. While in Toronto, she also worked in the area of chronic pain treatment. Her current work interest is obstetrical anesthesiology.
Since coming to Kingston, Dr. Haley has becoming involved in undergraduate medical education and has really enjoyed working with medical students, she said in an interview.
“When this [position] came up, it seemed to be something I’d be interested in, helping students beginning their path to success.”
She noted that her own career has included a variety of experiences, including being a peer assessor at CPSO and sitting on a number of OMA committees.
“I’d like to share the perspective of medicine that involves doing other things besides practicing medicine on a day-to-day basis,” she added.
For appointments with Dr. Haley or any of the Student Affairs advisors, please contact firstname.lastname@example.org , stop by the Student Affairs office in the Undergraduate Medical Office or call the Learner Wellness Assistant at 613-533-6000 x78451.
Wrapping up case-based learning sessions effectively
We often spend a lot of time planning our classes, especially our case-based small group learning (SGL) sessions. We tailor our sessional learning objectives to the course objectives that have been assigned, selected solid preparatory materials, build great cases and craft meaningful questions for groups to work through.
This makes sense, as the small group learning (SGL) format used in Queen’s UGME program is modeled on Larry Michaelsen’s team-based learning (TBL) instructional strategy that uses the majority of in-class time for decision-based application assignments done in teams.
One comment we often read on course evaluation forms and hear directly from students, however, is that sometimes they walk away from an SGL session and still aren’t sure what’s important.
Much of the focus in the literature on TBL is on the doing – setting things up, building great cases, asking good questions to foster active learning. There’s not as much written about how to finish well.
Wrapping up your SGL session should be as much a planned part of your teaching as preparing the cases themselves. If you build the time into your teaching plan, you won’t feel like you’re shouting to learners’ backs as they exit the classroom, or cut off as the next instructor arrives. Nor will you find yourself promising to post the “answers” to the cases on MEdTech. Sometimes it’s not the answers that are important, but the steps students take to get there.
Wallace, Walker, Braseby and Sweet remind us that the flipped classroom we use for SGL (preparation before class, application in class) is one “where students adopt the role of cognitive apprentice to practice thinking like an expert within the field by applying their knowledge and skills to increasingly challenging problems.” One such challenge is figuring out what the key take-away points are from an SGL session. With this in mind, it’s a good idea to plan your session summary, but then have students take the lead since “the expert’s presence is crucial to intervene at the appropriate times, to resolve misconceptions, or to lead the apprentices through the confusion when they get stuck.”
So, have your own summary slide ready – related to your session objectives – but keep it in reserve. In keeping with the active-learning focus of SGL, save the last 10-15 minutes of class to have the groups generate the key take-away points, share them, and fill in any gaps from your own list.
Here’s a suggested format:
- Prompt the groups to generate their own study list: “Now that we’ve worked through these three cases, what are the four key take away points you have about this type of presentation?”
- Give the groups 3-4 minutes to generate their own lists
- Have two groups share with each other
- To debrief the large group, do a round of up four or five groups each adding one item to a study list.
- Share your own list – and how it relates to the points the student raised. This is a time to fill in any gaps and clarify what level of application you’ll be using on assessments.
- If you’d like, preview an exam question (real or mock): “After these cases, and considering these take-away points, I expect that you could answer an exam question like this one.” This can make the level of application you’re expecting very concrete.
Why take the time to wrap up a session this way? Students often ask (in various ways) what the point is of a session. With clear objectives and good cases, they should also develop the skills to draw those connections themselves. This takes scaffolding from the instructor. As Maryellen Weimer, PhD, writes in Faculty Focus, “Weaning students from their dependence on teachers is a developmental process. Rather than making them do it all on their own, teachers can do some of the work, provide part of the answer, or start with one example and ask them for others. The balance of who’s doing the work gradually shifts, and that gives students a chance to figure out what the teacher is doing and why.”
If you would like assistance preparing any part of your SGL teaching, please get in touch. You can reach me at email@example.com
 Wallace, M. L., Walker, J. D., Braseby, A. M., & Sweet, M. S. (2014). “Now, what happens during class?” Using team-based learning to optimize the role of expertise within the flipped classroom. Journal on Excellence in College Teaching, 25(3&4), 253-273.
Enjoy these early and lasting gifts from the Bracken Health Sciences Library
By Suzanne Maranda, Head, Bracken Health Sciences Library
When I meet faculty in person, especially if I’ve not seen them in a while, or if they are new to Queen’s, they often embarrassedly admit that they never come to the library. Over the years, I’ve refined my answer: ”Oh, but you do; you probably just don’t know it. Most links to full-text articles would not work if the Library had not done the behind-the-scenes work.” Medical students are also quite amazed to find out, during their first session of medical school, that a single annual journal subscription can cost more than their tuition! The Queen’s Library spends over $9 million annually on library resources, most of which are electronic. The proportion in the health sciences is among the highest, with well over 90% of the purchases allocated to online materials.
The materials purchased by this library have also changed over time. It used to be that books and journals were the only information sources for serious learning and research. In recent years, in addition to conventional books and journals, with many more online than in print, you may find, among others, point-of-care tools such as Dynamed and BMJ Best Practice, anatomy software and image banks, clinical skills videos, clinical cases, and DVDs ( the latter can be borrowed to show in class or recommended to students).
While the Canadian dollar was still strong, the Library made strategic purchases of journal backfiles, allowing perpetual online access to older journal content. Most of this electronic content is linked to PubMed and Medline and the other databases in the OVIDSP interface for seamless access to full-text.
Tip #1: After completing a database search, it is best to NOT use the “limit to full-text” option in OvidSP because that limit only retains the journals purchased via this interface provider or where it has an agreement with particular publishers. There are MANY more journals that we purchase from other vendors, but the links will display only after clicking on the “Get it at Queen’s” button.
We are also very pleased that the links to full-text have finally been implemented in PubMed! Tip #2: For the links to appear, you must link to PubMed from the Bracken Library homepage (look under Find Articles). When you click on a citation, you will see this link:
in the top right corner, sometimes in conjunction with the publisher’s link. The Queen’s links will let you know exactly what years of the journal were purchased and, if the desired article is unavailable in full-text, you will see a link to order it from our Interlibrary Loans (ILL) service.
This brings me to an important change that will go into effect early in January 2015. All health and life sciences faculty and students will be able to order interlibrary loans using RACER. This service allows you to place orders and keep track of them yourself, but more importantly, it is linked to a desktop delivery system. Requested articles will be delivered as a link embedded in an email message. Remember that the Library no longer charges for interlibrary loan requests. More information will be sent to all health sciences faculty in December.
Course Reserve: Another service has changed this fall: there are now other options to place items on Course Reserve. Faculty have always been able to request that books or print journal articles be placed on reserve for students to sign out. These items are to be highly used by the entire class, and the reserve function allows for very short loans, usually 3 hours, which ensures that the entire class can have access within a reasonable amount of time. This is still the only way to handle a complete print book, but what about a chapter? Or an electronic article? Many faculty now put links to course readings in MedTech Central, and maybe we can help:
Tip #3: Bracken Library staff can scan a book chapter or a journal article and send faculty a pdf file for upload to MEdTech Central. This also applies to existing online materials: a persistent link can be created, which insures that you are using a reliable link over time and that the item is accessible from off campus. Please send requests to firstname.lastname@example.org. Now is the time to plan for the Winter Term!
On behalf of the entire Bracken Library staff, please accept my best wishes for the holiday season and for a healthy and productive 2015.
Medical Student Debt:
Is it a problem, or just a shrewd investment?
By the end of his or her medical education, the average Canadian graduate will owe $71,721. That amount, which has increased by about 7.3% over the past 5 years, may seem either huge or trivial depending on your perspective and stage of life. Interpretation might be enhanced with a few more details:
- 17.5% manage to get through medical school with no debt at all, a figure that has not changed over the past 5 years.
- on the other extreme, 6.2% report debts of over $200,000, which has increased from 4.1% in 2010
- 28.3% report already having debt before even entering medical school (26.4% in 2010), with an average premedical indebtedness of $7,465 or, perhaps more telling, an average of $27,094 for those who report any debt (comparing to $6,506 and $25,968 respectively for 2010).
- 32.5% report having accumulated “non-educational debt” during medical school (eg. credit cards, car loans, mortages) averaging $23,976 (comparing with $31,455 in 2010)
- 28% feel that the amount of financial assistance available to them fails to meet their needs (compared to 31.5% in 2010)
- 3.7% report “no need for financial assistance” (unchanged over the past 5 years)
All this comes from the Canadian Medical School Graduation Survey, which is conducted as part of the Academy of American Medical Colleges Graduation survey and was completed (voluntarily) by 2,048 graduating students in 2014 including (I’m pleased to report) 99% of our graduating class at Queen’s.
At the same time, tuition rates appear to be on the rise (http://studymagazine.com/2011/11/01/tuition-canadas-medical-schools-rises/), currently averaging about $11,000 annually, but with considerable variability between schools, ranging from as low as about $4,000 to highs of over $25,000, according to the Canadian Medical Education Statistics published by the Association of Faculties of Medicine of Canada (http://www.afmc.ca/pdf/CMES2014-Complete-Optimized.pdf).
So, what does all this mean? Is this a problem that we and other medical schools need to engage, or are we observing what is, from a strictly financial perspective, an investment by shrewd and well- informed young people in an education that will lead to a secure, well-paying future in which they should be able to quickly dissipate even the higher levels of indebtedness?
On the “what’s the fuss” side of this argument are those who point out that medical students, once enrolled, have easy access to large loans from banks and other institutions who are confident in their success and financial prospects. They would note that almost all medical students graduate to lucrative careers (unlike most other university graduates), and that even a resident physician’s income, sensibly managed, provides the means after graduation to pay down those loans. They would further point out that there is very little post-graduation default on debts. Finally, they might make the point that these students are, in fact, adults who make conscious and highly informed career and financial decisions, and that medical schools are either their parents nor socially responsible for those decisions.
Those on the “we have a problem” side of this issue might make the following points:
- Although manageable after medical school and a minor issue in retrospect, the perception of increasing debt during medical school is a major stressor for students during an admittedly demanding period of their training, and may therefore distract from their education
- The high debt load may influence career decisions, prompting students to consider specialties with shorter duration of training and greater perceived long-term economic benefits.
- High debt load may discourage students from taking up research, educational or other academic training opportunities, either in parallel with or after their core training.
- The high costs and accumulated debt may discourage many young people from socioeconomically challenged backgrounds from even considering careers in medicine, thus establishing a further barrier to the social diversity that all medical schools and the medical education community are endeavouring to establish. To quote the AFMC’s Future of Medical Education In Canada: A Collective Vision for Medical Education in Canada:
“Achieving this diversity means attracting an applicant base that is more representative of the Canadian population. This will involve, for example, addressing perceived and real barriers to medical education, such as the high debt loads of medical graduates.”
The last point is particularly vexing. It’s easy to imagine that, for a family of limited financial means and with incomplete knowledge of the financial realities, the prospect of over $20,000 in annual tuition and possibly hundreds of thousands in accumulated debt may be sufficient to quash any dreams of medical education very early in life. (see previous blog article http://meds.queensu.ca/blog/undergraduate/?p=1165&preview=true&preview_id=1165&preview_nonce=e904b6e40f&post_format=standard).
In addition, there are considerable financial hurdles a student must face to simply apply to medical school, including three to four years of pre-medical undergraduate medical education, MCAT examinations (including preparation and travel) and quite likely a sense that income-generating jobs should be sacrificed in order to pursue studies or activities deemed more “attractive” to medical school admissions officers. Although considerable financial assistance and loans are available to students once accepted to medical school, there is no similar level of assistance to those in the application process where it would arguably be of greater benefit.
At Queen’s, we are concerned about the rising profile of student indebtedness and it’s impact on both current and prospective students. To further examine this issue, we have established an Advisory Panel on Medical Student Debt, chaired by Dr. Greg Davies and supported by Brian Rutz, UG Financial Officer. The panel is populated by several current students from all years, recent graduates in residency training, not-so-recent graduates now in the early years of independent practice, and several members of faculty and the university community with interest in this issue. That group has already begun its work by undertaking a review of the Canadian medical school environment through the Graduation survey, and current literature. It is focusing on several topics:
- The sources of debt
- The impact of debt on individual students
- Counseling and information sources available to students as they engage financial planning
- Financial aid availability and access
- How support might be provided to young people considering careers in Medicine
Their findings and recommendations, once available, will be brought forward for wide discussion and implementation. I’m sure Greg and his committee would appreciate hearing from readers about any and all of these issues. I know I would.
Anthony J. Sanfilippo, MD, FRCP(C)
Undergraduate Medical Education
WHAT’s NEW in the world of Learning Technologies ?
I recently had the opportunity to attend the DevLearn 2014 Conference.
The conference was about discovering tomorrow’s learning technologies, strategies and practices today and joining the community of industry pioneers that are exploring the new learning universe and are defining the future of training and development.
I jam packed my days with amazing learning sessions that I thought we as a team would get the most out of. One of which I thought would benefit all of us back at the office was Forty-five Free (or Cheap) Online Learning Tools in 45 Minutes: What many instructional designers may not know is that for every $1,500 tool, there’s a free or low-cost alternative that can do the job just as well. This session covered a selection of tools that are available today and have many of the capabilities of expensive applications that can decimate a budget.
Some of the free tools I found may come in handy include:
To download the free version visit windows marketplace
OneNote is your digital notebook for keeping track of what’s important in your life.
To download the free version visit Microsoft Marketplace
Inkscape is professional quality vector graphics software which runs on Windows, Mac OS X and Linux. It is used by design professionals and hobbyists worldwide, for creating a wide variety of graphics such as illustrations, icons, logos, diagrams, maps and web graphics. Inkscape uses the W3C open standard SVG (Scalable Vector Graphics) as its native format, and is free and open-source software.adobe illustrator, use for prep work.
To download the free version visit www.inkscape.com
Audacity – great tool to record and edit sound. To download the free version visit http://audacity.sourceforge.net/download/
HandBrake is a tool for converting video from nearly any format to a selection of modern, widely supported codecs.
To download the free version visit https://handbrake.fr/
To download the free version visit www.fotosizer.com
Delicious is a free service designed with care to be the best place to save what you love on the web. We keep your stuff safe so it’s there when you need it – always. Delicious remembers so you don’t have to. Delicious is a free and easy tool to save, organize and discover interesting links on the web. To download the free version visit https://delicious.com/
and the list goes on…
IRfanview – image editor, convert to any file format, edit – simple to use. Visit http://www.irfanview.com/
Windows Movie Maker used to make movies with images, videos and sound – Visit http://windows.microsoft.com/en-us/windows/get-movie-maker-download
TotalRecorder – capture any sound played by a computer (including streaming audio, Internet telephony, and PC games), and use the included time shift-feature for off-hours recording. Visit www.totalrecorder.com/
VLC is a free and open source cross-platform multimedia player and framework that plays most multimedia files as well as DVDs, Audio CDs, VCDs, and various streaming protocols. Visit http://www.videolan.org/vlc/index.html
Freesound is a collaborative database of Creative Commons Licensed sounds. Browse, download and share sounds. www.freesound.org royalty free music to use
Playlater is the first DVR for online video. Visit http://www.playon.tv/playlater
Sketchup easiest way to draw 3D drawings. Visit www.sketchup.com
Onedrive storage, keep your files and photos in onedrive.Visit www.onedrive.com
Join Me – screen sharing. Visit JoinMe
StoryboardThat online storyboard creater, powerful and easy to use. www.storyboardthat.com
7-zip is a file archiver with a high compression ratio. Visit www.7–zip.org/
Neobook create your own windows app (wysiwig) Visit Neosoftware.com
Open Source Windows – utility for manipulating archives. Formats 7z, ZIP, GZIP, BZIP2 and TAR are supported fully, other formats can be unpacked. Visit http://opensourcewindows.org/
ProjectLibre – open source (similar to microsoft project) gantt charts Visit www.projectlibre.org
Gspilt split 10 dvds into smaller ones to share, exe on the disc. Visit www.gdgsoft.com/gsplit/
Malwarebytes protects you from new online threats that antivirus can’t detect. Visit malwarebytes.org
Coursera is an education platform that partners with top universities and organizations worldwide, to offer courses online for anyone to take, for free. Visit www.coursera.org
Any Video Converter takes videos from your computer or downloaded from the Internet and converts them into just about any format you’d like. Visit http://www.any-video-converter.com/products/for_video_free/
Awesome Screenshot Capture the whole page or any portion, annotate it with rectangles, circles, arrows, lines and text, one-click upload to share. Visit http://awesomescreenshot.com/
Snagit Use images and videos to show people exactly what you’re seeing. Snagit gives you an easy way to quickly provide better feedback, create clear documentation, and change the way you work together. AVG Antivirus updates on a regular basis Visit http://www.techsmith.com/download/snagit/
Jump Desktop free remote access tool, anywhere you are (works through google) Visit https://jumpdesktop.com/
PowToon is the brand new Do-It-Yourself animated presentation tool that supercharges your presentations and videos! Save massive amounts of time and money by creating Presentoons that bring the WOW!-factor to your educational presentations, and much more. Visit www.powtoon.com
Doro PDF Writer installs a virtual printer on your system with which you can create PDF documents for free from any Windows app. Visit http://doro-pdf-writer.en.softonic.com/
FLVTO free conversion tool pull files from youtube. Visit http://www.flvto.com/
Red Kawa is a video converter. Visit http://www.redkawa.com
PDFtoword is a pdf convert it into work excel ppt etc Visit https://www.pdftoword.com/
Infogr.am is a tool to make infographics the easy way, create charts that are quick and easy to use and easy on the eyes. Visit https://infogr.am/
Otixo ties all your cloud drives together in one app. Visit www.otixo.com
Bitstrips is a tool used to turn yourself and your friends into cartoon characters, and create and share your own awesome comic strips. Visit www.bitstrips.com
Lunapic if you would like to create an image with a transparent background, upload an image to change to transparent (no download) Visit www.lunapic.com
Google Web Designer a tool to create engaging, interactive HTML5-based designs and motion graphics that can run on any device. Visit www.google.com/webdesigner/
Along with this useful session on Forty-five Free (or Cheap) Online Learning Tools, I went to sessions on the The Top 10 Authoring Tools of 2014 – and the Forecast for 2015, The xAPI—Liberating Learning Design, Building Interactive Slides in Storyline, Transform Users into Contributors: Kaplan’s Path to User-generated Content, xAPI Hyperdrive Showcase, The xAPI for the Non-developer, Demo Fest featuring eLearning Modules, and How to Make Community Part of Your Training.
If any of these topics interests you or you are thinking of exploring any of these tools, please contact Lynel Jackson 613-533-6000 x74919 E-mail: email@example.com
New Features on MEdTech
At the fall Curricular Leaders’ Retreat, Lynel Jackson highlighted four new and improved MEdTech features that can assist faculty in presenting information for students and in planning learning events and courses.
Adding Resources to Learning Events
The EdTech team has completely redesigned the way resources (such as files, links, and quizzes) are added to the Learning Events and displayed to learners in Student View. This new view uses much of the information the EdTech team has collected for years during the upload process, like “Should this resource be considered optional or required?” and “When should this resource be used by the learner?” then displays it to learners in a clear and user-friendly timeline on the Learning Event page. The new format clearly shows what learners need to do to prepare for class, and also clearly marks what resources are required versus what is for information only.
In Development: In the future, these classifications will be used to provide learners with a checklist on their Dashboard, identifying all the activities they need to complete before classes for the week.
The EdTech team has enhanced MEdTech’s Curriculum Explorer tool which is now able to show not only where objectives (at any level) are mapped to Courses, and Learning Events, but also Gradebook Assessments. Faculty members and staff can use this tool to really explore the curriculum at all levels.
There are a number of new and enhanced reports – such as MCC Presentations by Course, Course Objectives by Events Tagged, and Learning Event Types by Course – that can assist in evaluating past course iterations as well as planning the next one. Curriculum coordinators can generate these reports for Course Directors, on request.
One of the most frequently requested features by faculty has been the ability to easily upload images or documents, and embed video into rich text areas throughout the MEdTech platform. With this Fall release, the team was pleased to announce this can now be done within any of the rich text areas.
To upload images or documents, click the “Browse Server” button from within the “Image” or “Link” icons. This will open your personal “My Files” storage area where you can upload images or documents from your local computer. Once you upload the image or document, clicking it will embed the image or document directly in the rich text area. You can also embed video from the Queen’s Streaming Server, YouTube, or Vimeo into any rich text area by clicking the “Embed Media” icon, and pasting in the “Embed Code”.
For questions on these updates and other aspects of MEdTech, reach the Education Technology team at firstname.lastname@example.org
Everything you need to know about exam questions types in our curriculum!
Are all exam questions created equal? Not really—different type of questions test different levels of understanding. In the UGME program, we use a variety of exam questions to assess student learning—broadly classified as multiple-choice questions (MCQs) and short-answer questions (SAQs). But within these broad categories are a range of types of questions designed to test different levels of cognition. We use these different types of questions at different points both within courses and within the program.
Based on Bloom’s Taxonomy
Bloom’s taxonomy is a classification system used to define and distinguish different levels of human cognition—thinking, learning, and understanding. The taxonomy was first developed in the 1950s by Benjamin Bloom and further revised by him in the 1990s. In his original version, there are six levels of cognitive behaviours that explain thinking skills and abilities of learners. The original six levels of cognition as described by Bloom are: knowledge, comprehension, application, analysis, synthesis and evaluation. Educators have used Bloom’s taxonomy to inform or guide the development of assessment, such as with the construction of MCQs. MCQs are widely used for measuring knowledge, comprehension and application of learning outcomes. Our curriculum uses MCQs in different assessment formats, for different purposes, and those are described below.
You may hear acronyms and terms about assessment in our UGME program: RATs, MCQs, SAQs, Key Features. Here is a brief description of each:
Readiness Assessment Tests (RATs)
RATs used in our curriculum often consist of 10-15 multiple-choice questions that are linked directly to the readings (and/or prior lectures). A RAT focuses on foundational concepts that will be important for following SGL activities. MCQs found on a RAT, test for knowledge (i.e., recall information) and less for application of knowledge. Examples of verbs used in the question stem that would test knowledge include: define, list, label, recall, select, name, outline, or match.
Multiple-choice questions (MCQs): on midterms and finals
There are three components to an MCQ: the stem, lead-in question, and options that consist of one correct answer and typically three distractors (wrong answers). The stem should be directly linked to a learning objective assigned to a course. MCQs that are used on midterms and final exams often test for comprehension and application of knowledge; this is beyond the recall information that is typically the case with MCQs on RATs. Some multiple-choice questions may assess simple recall, depending on the learning objectives of the course but should be kept to a minimum. Verbs used in the question stem to test comprehension include: predict, estimate, explain, indicate, distinguish, or give examples. Verbs that would test application include prompts such as: solve, compute, illustrate, interpret, demonstrate, or compare.
Short-answer Questions (SAQs)
SAQs typically are composed of a case scenario followed by a prompt that requires a written answer that varies in length from one or two words to several sentences. SAQs often test the higher cognitive skills in Bloom’s taxonomy. Final examinations in our curriculum are typically composed of a mix of MCQs and SAQs. To test analysis, verbs in the question stem include: explain, arrange, select, infer, calculate, or distinguish. Verbs such as develop, design, plan, devise, formulate, or generalize test for synthesis, whereas verbs in the question stem to test evaluation include: argue, assess, estimate, justify, predict, compare, conclude, or defend.
Key Features Questions
Key features problems are used by the Medical Council of Canada for the assessment of clinical decision-making skills in the MCCQE Part 1. Key features problems have a case scenario usually followed by two or three questions, each question testing one or more key features. A key feature is defined as a critical step in the resolution of a clinical problem, and key-feature problems consist of clinical case scenarios followed by questions that focus only on those critical steps. While knowledge is an important feature for effective problem solving, the challenge posed by key features problems is the application of knowledge to guide clinical decision-making. For each question, instructions may require selection of whatever number of responses is appropriate to the clinical tasks being assessed, and there may be more than one response in the answer key. The development of key features problems for clinical decision-making is being piloted in the Clerkship curriculum courses this year.
How do we administer our tests?
Queen’s Undergraduate Medical Education has moved to an electronic exam system called ExamSoft for the administration midterms and final exams in Preclinical and the Clerkship curricular courses. Medical students no longer write exams on paper; rather they do it all on laptops. This greatly facilitates marking of exams, and it means we are no longer managing huge volumes of paper and deciphering student handwriting.
- Page, G., Bordage, G. & Allen, T. (1995). Developing Key-feature proglems and examinations to assess clinical decision-making skills. Academic Medicine, 70 (3).
- Laura April McEwen, OHSE 2011, MCQ Checklist
Brainstorming in the classroom
Have you ever used brainstorming in your teaching? If you want groups of students to come up with a variety of ideas quickly, brainstorming is one tried-and-true way to get creative juices flowing.
Since the concept was introduced in Alex Osborn’s 1953 Applied Imagination, brainstorming has caught on in business, education, volunteer organizations and elsewhere to generate ideas and solve problems.
Brainstorming, as set out by Osborn, is designed to produce a large quantity of ideas in a short space of time, in order to encourage creativity. He had four simple rules for brainstorming sessions:
- Don’t allow criticism
- Encourage wild ideas
- Go for quantity
- Combine and/or improve on others’ ideas
Last week I saw a post on Twitter that suggested “brainwriting” rather than brainstorming. I was intrigued and clicked-through, only to find a new name for a familiar best practice: brainstorming works best when it’s planned, not haphazard, and it starts with the individual, not the group.
As described by Patrick Allan (citing work of Leigh Thompson and Loran Nordgren) brainwriting avoids the brainstorming pitfall of anchoring: where an early idea streams all other suggestions in a particular direction. “Brainwriting” gives individual team members time to write down their own ideas free of others’ influences.
Osborn himself advocated this (although he didn’t use the term brainwriting), asserting that the best ideas come from a blend of individual and group work. Classroom brainstorming shouldn’t be unplanned: students should have prep and thinking time.
As Robert Sutton notes in “Eight Tips for Better Brainstorming”: “Skilled organizers tell participants what the topic will be before a brainstorm.”
Barbara Gross Davis also encourages individual preparation in Tools for Teaching. She suggests posing an opening question and having students spend five minutes writing a response. This “gives students time to think and enriches subsequent discussion.”
Here are some other ideas to encourage better brainstorming in your classes:
- Assign roles within the brainstorming group. Groups need a moderator (to guide discussion, keep the group on topic, and encourage wide participation), a scribe (or two) to capture the ideas (using either flip charts, Post-It notes, computers or consider audio recording), and members (to contribute and build ideas).
- If you’re going to use brainstorming, make sure your scribes have some tools. To Osborn’s original four rules for brainstorming, OpenIDEO adds be visual: “In live brainstorms we use coloured markers to write on Post-its that are put on a wall. Nothing gets an idea across faster than drawing it. Doesn’t matter how terrible of a sketcher you are! It’s all about the idea behind your sketch.”
- MindTools advises that the moderator can help keep the team on task and can help the team avoid narrowing its path too soon. “As the group facilitator, you should share ideas if you have them, but spend your time and energy supporting your team and guiding the discussion. Stick to one conversation at a time, and refocus the group if people become sidetracked.”
- Remember, the students who are the moderators and scribes aren’t actively brainstorming while they’re attending to their key roles. Encourage teams to share these tasks throughout a term, so it’s not always the same couple of people who end up taking notes rather than contributing their ideas.
And, what to do with all those ideas the groups generate? Sutton points out that brainstorming should “combine and extend ideas, not just harvest them,” so have a plan for what you want students to do next.
The next steps are sorting and follow-up. In Small Group and Team Communication, Harris and Sherblom recommend an “ACB Idea Sorting Method”:
- Assign an A to the best one-third of the ideas
- Assign a C to the least usable one-third
- The middle one-third automatically receive a B
- Go back to the B’s and separate them into the A or C category
- Store the C category ideas for later use
- Prioritize the A’s in terms of their importance, urgency, or applicability to the problem at hand.
The Education Team can help you with incorporating brainstorming and other techniques in your teaching. Contact us to arrange for one-on-one coaching or to facilitate a workshop for your team.
7 Tips on Better Brainstorming. (n.d.). OpenIDEO. Retrieved August 12, 2014, from https://openideo.com/blog/seven-tips-on-better-brainstorming
Allan, P. (n.d.). Use “Brainwriting” Instead of Brainstorming to Generate Ideas. Lifehacker. Retrieved August 12, 2014, from http://lifehacker.com/use-brainwriting-instead-of-brainstorming-to-generate-1615592703?rev=1407126541539&utm_campaign=socialflow_lifehacker_twitter&utm_source=lifehacker_twitter&utm_medium=socialflow
Brainstorming: Generating Many Radical, Creative Ideas. (n.d.). Brainstorming. Retrieved August 12, 2014, from http://www.mindtools.com/brainstm.html
Davis, B. G. (2009). Tools for teaching (2. ed.). San Francisco, Calif.: Jossey-Bass.
Harris, T. E., & Sherblom, J. (2011). Small group and team communication (5th ed.). Boston: Pearson/Allyn and Bacon.
Johnson, D. W., & Johnson, F. P. (2009). Joining together: group theory and group skills (10th ed.). Upper Saddle River, N.J.: Pearson/Merril.
Sutton, R. (2006, July 25). Eight Tips for Better Brainstorming. Bloomberg Business Week. Retrieved August 12, 2014, from http://www.businessweek.com/stories/2006-07-25/eight-tips-for-better-brainstorming
Incorporating technology into teaching should focus on providing high-quality learning experiences for students, not just adding the latest tech fad to your teaching toolbox.
That was one of the messages shared by Sidneyeve Matrix, PhD, keynote speaker at the 7th annual Celebration of Teaching, Learning and Scholarship in Health Sciences Education. Sponsored by the Office of Health Sciences Education, the theme of the one-day conference was “Learning Together: Relationships in Health Sciences Education.”
Matrix, a Queen’s National Scholar and Associate Professor with the Department of Film and Media, Faculty of Arts and Science, addressed the topic of High-Engagement and High-Tech Teaching and Learning Experiences, by Design.
Although most of today’s students have grown up with technology, they’re not all the tech experts some may expect. They have surface knowledge of technology they use, but not necessarily a broad range of skills. And while students may not have deep digital competencies, they expect faculty to have them, Matrix said.
The first step to enhancing teaching with technology is addressing the faculty tech-skills gap through faculty professional development, Matrix suggested. This, she acknowledged, may be easier said than done: the biggest barrier to tech adoption by both students and faculty is time.
So, why bother with educational technologies? The payoff in student learning has been studied: teaching with edtech and social media improves student outcomes by 10 percent. And what about the distraction factor? Another study Matrix cited revealed students with smartphones study 40 extra minutes per week versus those without them.
Matrix advised faculty interested in incorporating more technology in their teaching to seek out innovators within their own departments and schools: approach these people to find out what’s worked for them and what hasn’t. She said blended learning teams should include ITS consultants, instructional designers and faculty peer mentors. Key messages: don’t go it alone and don’t think you have to reinvent the wheel.
And, she emphasized, focusing on students’ learning experiences—not the technology—is the key to success. Like all good teaching, teaching with technology should focus on excellence and engagement, not just adding in a tech tool or two – or 20.
While there’s “choice abundance” in online tools for teaching—Matrix pointed out there are over 2000, producing “choice fatigue”—too much technology can turn a good course into a “Frankencourse”, producing frustration for all concerned and lower student learning outcomes.
Matrix also advocates incremental innovation, pointing to her own Film240 Media and Culture course: its first iteration in 2007 had 75 students; by 2009 it had 500 students and a social media component. In 2011 she added a new online section, mobile app and webinars and boosted enrolment to 1000. The 2013 class had 1400 students, e-flashcards, podcasts, eBook, self-quizzes and lectures available on demand. Her point: she didn’t do it all in one term, or even one year.
One technology-assisted assignment Matrix showcased in her presentation was infographic digital posters, used as an alternative to a research essay assignment. These are shared via the course Learning Management System (LMS) for peer-to-peer inspiration and feedback. Students can use Piktochart to create their assignment.
These aren’t just pretty posters, but well-researched assignments presented in a visually-appealing, accessible way. “It’s visual storytelling with research narratives,” she said.
* * *
What’s your favourite tech teaching tool? Let us know what it is and why it works for you by sharing in the comments.
If you’re interested in tech teaching training, let us know what topics are of interest to you. We’ll incorporate these requests in our future UGME faculty development planning.
Find the full slidedeck from Dr. Matrix’s presentation here. You can find more on trends in digital culture, communication and commerce, with emphasis on social, mobile, and educational technology at her Cyberpop! blog.