Bringing things into focus: Using focus groups to collect feedback
By Theresa Suart & Eleni Katsoulas
Amongst the plethora of student feedback we solicit about our courses, you may wonder why we sometimes add in focus groups. What could be added to the more than a dozen questions on course evaluation and faculty feedback surveys?
The information we gather in student focus groups doesn’t replace the very valuable narrative feedback from course evaluations, rather, it allows us to ask targeted questions, clarify responses and drill down into the data.
Developed from “focused interviews” around the time of the Second World War, focus groups emerged as a key qualitative research tool in the latter half of the 20th century. Robert K. Merton, a sociologist from Columbia University, is hailed as the “father of the focus group.” (He died in 2003 at age 92.)
Merton used focused interviews to gain insight into groups’ responses to text, radio programs and films. Politicians and marketing companies soon seized upon focus groups to gauge voter and consumer trends. The Queen’s UGME Education Team uses focus groups in a targeted way to augment information gleaned from course evaluation feedback, course director’s meetings with academic reps and other feedback tools.
According to a briefing paper from Carnegie Mellon University, focus groups are “particularly effective” for eliciting suggestions for improvement. “They are also much more flexible than surveys or scales because they allow for question clarification and follow-up questions to probe vague or unexpected responses.” It also helps that faculty rate focus groups as “accurate, useful and believable”.
If you’re asked to participate in a focus group, only agree if you think you have something to contribute to the investigator’s project or purpose. (Sure, it’s fun to come for the free food, but be prepared to contribute in a meaningful way).
- To be informed if the focus group is for research or curricular innovation (or both). Research studies must have approval from the Research Ethics Board and require specific paperwork to document informed consent. Curricular innovation focus groups are less formal, but will still respect confidentiality of participants. These might not have the same paperwork.
- The facilitator to set the ground rules, and guide the discussion. Savvy facilitators will do this with a minimum of fuss: they will listen more than they speak. (But you can certainly ask for clarification if you’re not sure of a question).
- A co-facilitator will likely take notes and monitor any recording equipment used. The co-facilitator may summarize after each question and solicit further input as required.
- You’ll be asked specific questions, and engage in conversation with the other participants.
What you shouldn’t expect:
- A venting session. This isn’t the time to just complain. A focus group is looking for constructive feedback and suggested solutions.
- To always have your say: the facilitator may realize they have reached saturation on a particular question and will move on. This is to respect your time. (You’ll have an opportunity to send additional comments electronically afterwards if you felt there is an important point that was missed).
What you can do to prepare:
- If the questions are provided in advance (this is best practice but not always possible on tight timelines!) you should take some time to think about them.
- Be sure you know where the meeting room is, and arrive on time.
What you can do during:
- Contribute, but make sure you don’t end up dominating the conversation. The facilitator will be looking for a balance of views and contributors.
- Listen attentively to others and avoid interrupting. The facilitator will make sure everyone has a chance to contribute – you’ll get your turn.
What you can expect from data collected at a focus group:
- It will be confidential. Different strategies are employed. For example, you may be assigned a number during the focus group and participants asked to refer to people by number (“Participant 2 said…”).
- In a formal research study, you should be offered an opportunity to review the data transcript after it is prepared. (This is sometimes waived on the consent form, so read carefully so you can have realistic expectations of the investigator).
- The end product is a summary of the conversation, with any emergent themes identified to answer the research questions.
What you can’t expect:
- A magic bullet solution to a challenge in a course or class.
- One hundred percent consensus from all participants – you can agree to disagree.
- For all outlier opinions to be represented in the final report. These may be omitted from summary reports.
We’re always grateful to our students for donating their time to our various focus group requests throughout the year. These contributions are invaluable.
If you think this type of data collection could be useful in your course review and revisions, feel free to get in touch. It’s one of the tools in our qualitative research toolbox and we’re happy to deploy it for you as may be appropriate.
Eleni Katsoulas firstname.lastname@example.org
Theresa Suart email@example.com
Student wins prize for project on physicians with disabilities
What started as a project for her Critical Enquiry class turned into an award-winning poster presentation for Kirsten Nesset of MEDS 2017.
Nesset attended the 24th annual History of Medicine Days Conference at the University of Calgary in March where she won Best Poster Presentation for “Physicians with Disabilities in Canada: History and Future”.
Classmates Elena Barbir and Sophie Palmer also attended the conference, presenting on their Community-Based Projects. The three received the Boyd Upper Prize, which is awarded to the Queen’s medical student or students who have conducted original historical research and then had the work accepted for presentation at a peer-reviewed meeting.
Nesset’s interest in the area of disability started at home, she explained in an interview.
“It was something I was really interested in because my father has a visual disability and he’s an engineer,” she said. “He lost his vision when I was about 10 – so I grew up with him adapting to that and his work making accommodations.” And this got her thinking.
“You don’t really see many people with visual or physical disabilities in medicine and I wondered what the accommodations might look like for them and what kind of policy might be in place if there was any,” she said. “I wondered what that looked like in Canada.”
She quickly discovered that there wasn’t much information readily available. “It ended up being a much more global project in the end because there’s very little research in Canada,” she said.
As her CE Mentor, Jacalyn Duffin, pointed out: “Her first discovery was that almost no one had published on that topic, although there was a robust literature on burnout, stress, addictions and other mental problems.”
“The absence of any historical predecessors meant that she had to do some original digging, to produce what is effectively the first history on the topic and to try to explain why the question has not been asked before,” Duffin added. “Her research involved searching the literature, news reports, and eventually interviews.”
“Although Kirsten’s focus was Canada, she discovered that a relative silence on physicians with disabilities pervades the literature in general, making her findings relevant well beyond our borders,” Duffin said.
Nesset has plans to continue research in this area. To start, she plans to interview some physicians through the Canadian Association of Physicians with Disabilities. “Some physicians have come forward to say they would be interviewed – because there isn’t a lot of narrative from Canada yet.”
She would also like to delve further into what medical schools list as technical requirements for graduates. “Part of my project was looking into admissions requirements and there’s nothing in those but there’s a lot of talk about meeting technical standards and technical requirements and each school approaches that differently,” she said.
As she is starting her clerkship rotations in the fall, Nesset is hoping to complete some interviews by the end of the summer, but sees this as a longer-term project.
“Realistically, this is something I’ll carry through the next year and hopefully finish up part-way through clerkship.”
One strong lesson from this project is that history does not necessarily mean antiquity or even a few hundred years ago, Nesset said. “From my experience, history can also be incredibly recent. I looked at history as of 1980, essentially, or 1975. Then up until now, which is why it’s titled ‘history and future’.”
“A history of medicine project doesn’t necessarily mean you’re looking far back in the past, it can be more recent and you can apply it to future considerations, for example for policy development,” she said.
We’d like to feature news about our students’ achievements at conferences such as this. If you have a suggestion for a student to feature in a future blog post, please email me at firstname.lastname@example.org. We’ll follow-up on as many as we can.
Thank you, Peter
The history of career counseling in our medical school divides nicely into three “eras”. Before 2006, students were informally supported through the efforts of faculty mentors, but there was essentially no structured program or standard approach. The next eight years or so can be rightfully dubbed the “Peter O’Neill Era”. Recruited to the role of Director, Career Counseling in July of that year Peter resolutely went about developing a program of individual counseling and innovative organized group activities that supported hundreds of students through the increasingly complex and stressful process of selecting and engaging a postgraduate residency position. He worked alone for much of that time, until joined by Dr. Kelly Howse about three years ago. As he steps down from that role this year and we enter the “post-Onellian” era of Career Counseling, it’s clear that he has provided our school with a very solid foundation to build upon.
The core of Peter’s approach and our current program is personalized counseling with attention to the needs of each student. That individual approach is enhanced with a series of information and orientation sessions that feature themes as diverse as CV preparation, how to dress and interview for success and how to navigate the CaRMS application process. Over the years, our students have had enviable success in the matching process, a testimony to Peter’s efforts. Kelly Howse has been able to build on the foundations Peter established, and is now leading the development of a national document on Career Counseling standards and best practice, which is rooted in many of the principles and practices Peter established.
I personally feel very grateful for Peter’s presence in our school. As a devoted Queen’s grad with experience in community practice before engaging his current specialty of Obstetrics and Gynecology, he brought a unique blend of personal dedication and practical “real world” perspective to his practice and teaching, as evidenced by numerous teaching recognitions through the years, including the very prestigious Connell Award and the Association of Academic Professionals in Obstetrics and Gynaecology of Canada (APOG) Educator of the Year Award. Moreover, he has always been willing to contribute and to serve, and I have certainly benefitted from his advice and support over the years. Although he’s moving on to another career role, I know he’ll remain dedicated to our school, and his life’s journey in education is far from complete.
What makes folks like Peter so very valuable to medical schools is really very simple: they truly and deeply care. That caring begins with their approach to the practice of medicine, but extends naturally to their students and their institutions. Great medical schools are built around such people.
Thank you Peter.
Anthony J. Sanfilippo, MD, FRCP(C)
Undergraduate Medical Education
June Curricular Leaders Retreat held: EPAs, Remediation and Feedback, oh my!
After bringing another busy semester to a close, UGME curricular leaders took time to reflect on the past year and take part in workshops and discussion groups on a number of areas of the curriculum at their semi-annual Curricular Leaders Retreat on June 19. The aim of the retreat was to share information and to generate ideas and solutions to address teaching and assessment challenges.
In his end of year report, Associate Dean Anthony Sanfilippo highlighted accomplishments of the past year as well as announced new faculty appointments.
After providing an entertaining and informative review of the process of curriculum renewal that UGME has undergone over the last several years, including the development of the “Red Book” objectives, Dr. Sanfilippo discussed how the emerging use of Entrustable Professional Activities (EPAs) will relate to and refine our existing curriculum and assessment processes.
Dr. Sue Moffatt presented an information session on how the three classroom-based “C” courses relate to both clerkship and the rest of the curriculum.
In a discussion about Service-Learning, led by Dr. Sanfilippo, faculty brainstormed ways additional service-learning opportunities could be created for medical students and others as well as ways they could support and encourage students in these endeavours. The Service Learning Advisory Panel will consider their suggestions and recommendations.
As a follow-up to last year’s popular workshop on remediation strategies, Michelle Gibson, Richard Van Wylick and Renee Fitzpatrick presented “Remediation 2” with additional cases and strategies.
For the afternoon, participants chose between a session on writing narrative feedback or one on making ExamSoft work for you.
Designed in particular for faculty working in clerkship, clinical skills and facilitated small group learning (FSGL), for the workshop on narrative feedback, Cherie Jones and Andrea Winthrop provided concrete examples and solutions to situations faculty routinely encounter when needing to provided constructive feedback to students. This included a discussion of ways in which oral and written feedback differ.
In the ExamSoft workshop, Michelle Gibson, Eleni Katsoulas and Amanda Consack worked with faculty to show how to tag mid-term and final assessments to match to assigned MCC presentations and Red Book objectives as well as coding for author and key word. Using these ExamSoft tools upfront makes it possible to use built-in reports to blueprint assessments, rather than having to do so manually. (For more on ExamSoft, check out the team’s poster from CCME at this link.)
To wrap up the day’s activities, pre-clerkship and clerkship course directors brainstormed with competency leads for ways the milestones identified for these intrinsic roles can be met throughout the curriculum. How to highlight and incorporate patient safety in different courses was also considered.
Documents from the Retreat are available to curricular leaders under “Retreats” on the Faculty Resources Community Page.
Bridging the gap between theory and practice in Medical Education: Entrustable Professional Activities.
Anyone who’s struggled through high school or university language courses will have observed, perhaps with exasperation, how young children learn to speak those languages quite effectively without the benefit of formal instruction. Growing children blissfully bypass linguistic theory and grammatical rules, and simply start speaking the language, employing a combination of imitation and trial-and-error to find what sounds and phrases produce desired effects. In fact, they may not even be aware that any linguistic conventions or grammatical structures exist. In my high school experience, French-speaking classmates had difficulty passing high school French courses because, as our non-Franchophone French teacher explained, the course was about the French language, not about speaking French. This distinction, lost on myself and the other peri-pubescent males of our rural Ontario community, caused many to abandon all hope for the educational system. There was a gap, it seemed, between formal “book learnin’” and real world skills that would allow people to function effectively and earn a living.
Every discipline, occupation or societal role can be regarded as requiring both theoretical underpinnings and practical application.
The theoretical components consist of the relevant knowledge base and a deeper understanding of the principles on which that knowledge base is established. This may involve learning scientific or abstract disciplines that might seem quite removed from the practical application. Such learning usually resides formally within our educational institutions and is recognized through the granting of diplomas or degrees.
Practical application, in contrast, is pragmatic, workplace-based and performance driven. Knowledge acquisition is more directly related intended purposes, and the ultimate goal is mastery of the specific skills, acts or functions understood to be requisite to the role.
The history of medical education is a story of struggle to balance theory and practice. Initially, medical education was purely a workplace, apprenticeship-based experience. Aspiring doctors worked with established practitioners and at some point, usually established by mutual agreement, were deemed ready to practice independently. The emergence of professional societies provided some external scrutiny and certification of competence. It was the rather profound intervention of the Carnegie Foundation and its sponsorship of the Flexner enquiry and subsequent report released in 1911 that moved medical education firmly into the university setting and established the requirement for fundamental education in the scientific foundations of medical practice.
Today, medical schools continue to struggle with establishing the appropriate balance between theory and practice. Educators ponder the degree to which fundamental science should be provided, the methods in which it should be taught, when and how patient-based experiences should be introduced. Students struggle to find “relevance” in their educational experience, particularly in the early years. A degree of mutual trust is essential to the process.
The emergence of “competency- based” education over the past decade or so is a valiant attempt to bridge the theory/practice gap. The “competencies” are based on the “roles” considered essential to (and characteristic of) the effective, practicing physician. In addition to expertise in clinical medicine and its scientific foundations, communication, collaboration, scholarship, advocacy, leadership and professionalism have been widely and rightfully accepted as attributes of the effective practitioner. Utilizing those attributes as a basis for development and design of a medical educational program may seem logical and appealing. However, on closer inspection, this extrapolation makes two key assumptions that are fundamentally flawed and have resulted in considerable challenges to our programs:
The first flawed assumption is that all competencies can (and should) be taught and learned. Many of the competencies relate to personal attributes, values or qualities. Examples drawn from our own competency framework include:
- The graduate is able to identify honesty, integrity, commitment, compassion, respect and altruism
- The graduate demonstrates respect for patient confidentiality, privacy and autonomy
- The graduate demonstrates respect for diversity, regardless of social, cultural or ethnic background
- The graduate demonstrates engagement in effective and shared decision making
Such objectives can be identified, characterized, used for purposes of selection and even required as a behavioural expectation. However, they are, for the most part, inherent characteristics that can’t truly be “taught”. It’s no more reasonable to expect that any individual can be taught to be a doctor than it is to expect than anyone can be taught to be a star athlete. Certainly good education can characterize the key expectations, contextualize their role and refine their application, but they cannot be developed de novo, regardless of good intentions, diligence and excellent teaching methods. Nonetheless, medical education programs devote precious curricular time and resources in attempts to ensure students possess attributes that, many would argue, should be substantially in place on admission.
The second flawed assumption is that all competencies can be reliably assessed. The medical education community has developed impressive expertise in the assessment of knowledge, skills and even complex tasks. However, the assessment of personal qualities such as interpersonal collaboration, compassion and integrity has not progressed much past the “know it when I see it” stage.
All this has led to a strong sense among both teaching faculty and students that there continues to be a “missing link”, essentially a theory/practice gap between the stated objectives of our program, and the fundamental goal of producing graduates able to excel as postgraduate program trainees and as young physicians.
To address these concerns, the medical education community is beginning to embrace an approach originally proposed by Dr. Olle ten Cate (ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005: 39(12); 1176, and ten Cate O. Trust, competence and the supervisors role in postrgraduate training. BMJ 2006; 333(7571);748).
“Entrustable Professional Activities” have been defined as units of professional practice. As such, they are tasks or responsibilities that trainees are to be able to perform independently by the time they complete their educational program. Importantly, EPAs are independently executable, observable, and measurable. In short, they go beyond what our students should know and be, and articulate in specific terms the things we expect our graduates to be able to do, and form the criteria by which they can be objectively and reliably assessed. The various competencies are necessary components required in order to achieve that EPA. However, demonstration of ability to perform the EPA, not the components, is the necessary final step to qualification.
To illustrate, let’s consider the simple and familiar example of driving a car, which can be considered a societal EPA. In order to achieve that EPA, candidates must master certain competencies, such as understanding the rules of the road, good vision, trustworthiness. Those attributes must be demonstrated or mastered in order to qualify to drive, but the ultimate “test” is the driving test itself.
An example of what might constitute a EPA for medical education might be the ability to perform a history and physical examination appropriate to patients presenting with certain key clinical presentations. That particular activity requires a number of requisite competencies including, for example:
- An understanding of the structure of the normal human body
- An understanding of structural changes that occur in various disease states
- An understanding of the symptoms and signs expected that are relevant to various presentations
- An understanding of the pathophysiologic mechanisms of clinical signs and symptoms
- An ability to communicate effectively with patients from various backgrounds
- The ability to maintain patient confidentiality
- The ability to interact effectively and respectfully with patients and their families
- The ability to understand the clinical utility and predictive value of various physical examination findings
- The ability to manage an interview effectively and efficiently
It becomes apparent from this list that medical expert, communicator, professional and scholar competencies are all required in order to carry out this particular, key EPA. They must all be learned and mastered individually, to be sure. However, individual achievement of each component is insufficient unless they “come together” to enable the learner to perform the fully formed professional activity.
Importantly, EPAs can be developed relevant to the fully qualified physician, and then described relevant to various stages of development. They therefore have the potential to unify the medical education continuum from entry to independent practice readiness.
A number of key organizations either have developed, or are in the process of developing EPAs. The American Academy of Medical Colleges sponsored an international consensus panel that produced a particularly attractive set of 13 EPAs currently being piloted at selected sites. The Association of Faculties of Medicine of Canada, at the suggestion of the Undergraduate Deans, has recently established a committee under the leadership of Dr. Claire Touche that is exploring the development of a common set of EPAs that could be utilized by all Canadian medical schools. The Royal College of Physicians and Surgeons of Canada, is incorporating EPAs as foundational component of it’s Competency by Design approach to postgraduate and continuing medical education.
In the meantime, our schools are likely to engage EPAs more actively as they endeavor to ensure their curricula are relevant to their students, and reliably address the real needs of postgraduate programs and society.
EPAs are the bridge that will take us from theory to practice.
Anthony J. Sanfilippo, MD, FRCP(C)
Undergraduate Medical Education
Baseball players, Cardiologists, and the art of Decision Making
What do great baseball players and cardiologists have in common? Not much, may be your first reaction. However, as I was preparing some comments on the topic of decision making for our clerkship class recently, I came to recognize some intriguing parallels.
Baseball players come basically in two varieties, pitchers and batters. Pitchers are large, powerful people who stand on a mound of elevated dirt and hurl the ball toward an imaginary area of space 60.5 feet away called the “strike zone”. The strike zone is defined by home plate and, believe it or not, anatomic dimensions of the batter. If the pitcher is able to do so three times, he/she records an “out”, and once sufficient outs are recorded, the game ends. It’s basically that simple. The rest is largely spitting and scratching.
The objective of the batter is to intercept the ball as it travels through the strike zone. To make this challenging, the batter must do so by swinging an implement remarkably ill suited to the purpose called a “bat”, which is a carved wooden stick barely wider than the ball. To make things even more interesting, the bat has a curved surface, which causes the ball to careen in virtually any direction unless very precise contact is achieved. Batters are also big, powerful people. When they get the bat into the right place at the precisely right time and connect with a rapidly moving ball, the result is a graceful, glorious flight through the stadium and into the stands. They are then heroes and the focus of much jubilation and adulation. When they swing and miss, they look rather ridiculous, even comical, and are the target of derision and amusement from the assembled masses. It’s truly all or nothing.
Now let’s do some simple math. An accomplished professional baseball pitcher can throw a baseball in excess of 90 miles per hour. The ball will therefore reach the strike zone 60.5 feet away in about 400 milliseconds, and will actually be in the strike zone and available to the batter for only about 5 msec. Neurologic activation of the various muscle groups required for the batter to even begin to move the bat takes about 200 msec, and then must be moved through the strike zone. All this means that the batter must commit to swing shortly after the ball leaves the pitcher’s hand. In fact, it’s been estimated that the batter must decide and commit to the swing within the first 100 msec of the ball’s flight. If they wait until the ball is closer, it’s far too late to hit the ball. In short, they must make a critical decision with minimal information, and must commit fully to that decision if they’re to have any hope of getting the ball into the field of play. There will always be considerable uncertainly (good pitchers can vary the speed and path of the ball), and they must be prepared to deal with inevitable failure. The greatest hitter of all time was, arguably, Ted Williams, whose lifetime success rate was about 40%. Even very good professional hitters fail 70 or even 80 percent of the time. What makes someone willing to take on such a task? What makes someone able to succeed? Much has been written about vision, reflexes, flexibility, swing speed. I would submit that great batters have two key and indispensible qualities:
- the ability to make and commit completely to rapid, mostly intuitive decisions unsupported by complete information, and
- the ability to deal emotionally and recover from failure.
Pitchers must also make decisions, but can control the pace of those decisions. They take time to consider each pitch, often delaying the game by wandering around the mound, rubbing the ball in a contemplative way, “adjusting equipment” and even conferring with teammates. They therefore have the ability to consider their decision, commit completely to that decision, and, only then, execute the decision. In short, they can become sure of what they wish to do and separate the thinking from the execution, unlike the batter who must do both virtually simultaneously.
So what’s this all have to do with Cardiology? When I began my career, cardiologists all did essentially the same things. Today, there are a number of sub-specialties within the field of Cardiology, but they divide basically into those who are primarily engaged in procedural (“interventional” or “invasive”) work and those who are not. Interventional Cardiology, whether it’s coronary or electrophysiologic applications, certainly requires the acquisition of key technical skills. But I would argue that the defining, key characteristics of these folks are their ability/desire/comfort in making critical decisions “on the fly”, often without full information, and being able to deal with sub-optimal outcomes. Those sub-optimal outcomes are fortunately very rare, unlike our baseball-playing friends, but may have considerably more serious consequences. Successes or “home runs” in the interventional world are really the norm, which is wonderful, but doesn’t change the critical nature of each individual decision. “Non-invasive” cardiologists also make critical, life determining decisions but, like baseball pitchers, have the option of doing so at a more controlled pace, and only after accumulating what they consider to be complete information.
Cardiology is, in fact, a microcosm of modern medical practice in general. Specialties and sub-specialties vary greatly as to procedural mix, how decisions must be made, and likelihood of dealing with adverse outcomes. As our students grapple with decisions about careers, they seek much information about the various specialties and disciplines available to them, but often pay scant attention to their own personal preferences and attributes so critical to career choice and professional satisfaction. Some key questions for medical students as they consider career options:
- How do you prefer to make decisions? Some prefer to gather information, consider alternatives, weigh potential outcomes and come to deliberate decisions, while others are comfortable with (and even prefer) more urgent situations where it’s necessary to make the best choice from the information available at the moment. Disciplines and practice settings differ with respect to the types of decisions that are required or appropriate, and therefore provide opportunities for various personalities. In my experience, students who choose surgical specialties do so early in their medical school experience and seldom deviate. Those engaging Internal Medicine usually don’t come to a final decision until late and only after serious consideration of numerous other options. Hardly surprising.
- Do you prefer continuing relationships with your patients, or situational, acute intervention? Although all medical disciplines are centred on the patient relationship, for many these involve problem-oriented and self-limited encounters. The preference for and and comfort with continuing relationships that are central to specialties such as Family Medicine can’t really be learned or acquired through practice.
- Are you comfortable with the inevitable adverse outcome that can occur despite what appeared to the right decision, appropriately applied? Every physician must learn to deal with these situations, but not all are able easily to move on without being personally affected, or developing self-doubt that may compromise subsequent performance.
These are personal, intrinsic, “hard-wired” qualities that can’t be learned, trained or denied. The key to making effective career decisions is self-awareness, and the way we make decisions is, itself, a key component of that awareness. Medical undergraduate programs are becoming increasingly aware of the need to provide students with the information and counseling they require. Here at Queen’s, we’re fortunate to have an excellent team in Student Affairs, including Drs. Kelly Howse, Susan Haley and Renee Fitzpatrick, who both develop learning events and meet with students individually to assist with career choice.
So, to get things started, are you a pitcher, or a batter?
Anthony J. Sanfilippo, MD, FRCP(C)
Undergraduate Medical Education
Medical grad receives Queen’s University Agnes Benidickson Tricolour Award
One of the highlights at Convocation on May 21 was the admission of one of the Meds Class of 2015 to the Queen’s University Tricolour Society.
Benjamin Frid was admitted to the Society through the Agnes Benidickson Tricolour Award—the highest honour given to a Queen’s student for non-academic, non-athletic activities. Recipients are chosen by their fellow students.
For Frid, it had been a bit of a wait to be inducted into the Tricolour Society: He was actually nominated and accepted for the award in 2012-2013, but it is presented upon graduation.
The award—named after Dr. Agnes Benidickson, Chancellor of Queen’s University from 1980 to 1996—is presented in recognition for valuable and distinguished service of outstanding individuals to the University. According to the Tricolour website, “such service may have been confined to a single field, or it may have taken the form of a significant contribution over a wide range of activities.” For Frid, his contributions definitely spanned a range of activities. Among those contributions included in the citation read at convocation were:
- He founded the Kingston chapter of Making Waves, a student-run organization that provides affordable private swimming lessons for children with disabilities
- He was Aesculapian Society president
- He formed of a wellness committee to address mental health issues for medical students
- He was founder and president of the Health Care Management Interest Group, a team involved with addressing the deficit in financial literacy that many physicians today are burdened with
“Ben’s spirited inclusive, and enthusiastic approach to life has influenced the lives of innumerous students and the greater Kingston community for the better,” the citation said.
Frid’s journey to this award actually began with his first undergraduate degree where he had what he describes as “limited extra-curricular involvement.”
“It left me feeling that I had really missed out on a lot of interesting and important opportunities,” he wrote in an email interview. “I think university is the perfect time to start becoming more involved. You are surrounded by such energetic people and a university that wants to help students do great things, I really think it’s the best time in person’s life to try and make a difference and improve the lives of those around them.”
Frid got more involved at the Telfer School of Business at the University of Ottawa where he started the Ottawa Making Waves chapter, was a teaching assistant and began taking leadership courses. This new habit of involvement continued when he came to Queen’s School of Medicine.
“At Queen’s, I was heavily involved in student government through our class council, our Aesculapian Society, and the Canadian Federation of Medical Students (in addition to lots of other groups and projects), but by far my favourite was Making Waves!”
Frid admits that balancing extra-curricular activities with medical school studies wasn’t always easy. “I had to learn some new skills and become a more organized person,” he said. “Fortunately, Queen’s faculty are very supportive of students who want to be involved,” he added.
“I think extra-curriculars are an important component of mindfulness,” he pointed out. “Just like eating well and exercising regularly, finding consistent positive and rewarding experiences are a key part of managing the heavy workload of medical school.”
“Even though it can create a bit of a time crunch, I think I was a far better medical student for the extra responsibilities I took on.”
“The beginning of medical school should not be the end of your hobbies and passions,” Frid said when asked for advice for the incoming Class of 2019. “Grow them! Pursue what you have loved to do, and take advantage of all the new experiences that will soon present themselves. Your fellow medical students are every bit as passionate as you are, and together you can do incredible things.”
Frid noted that he is “inspired by the people I have had the privilege of working for,” and pointed to one example from the early days of Making Waves in Kingston.
“I remember wondering how long it would take for instructors and their kids to bond, and for us to start seeing evidence of value we were generating for the community,” he said. “While setting up for just the second lesson, I remember watching one of our kids recognizing his instructor in the aquatic centre lobby, his eyes opening wide as could be, and him launching into a full speed sprint with arms outstretched to go hug his instructor he had only met one week before. I knew then that we had happened upon something special and that memory has stayed with me.”
Frid will begin his Family Medicine residency in July here at Queen’s. As he moves on to the post-graduate program, he’s left Making Waves in good hands. “The medical students in the classes of 2017 and 2018 are doing an incredible job of growing Making Waves from where we left off, and are to be commended for their hard work and successes,” he said. “Making Waves Kingston is a Queen’s-wide initiative with key leadership from the Queen’s School of Medicine, and it is thriving under its new leadership.”
According to the Society’s web page, Frid is the first medical student to receive the Tricolour since Ahmed Kayssi (Meds2009) in 2005-2006. Because of this, Frid was “particularly proud to be attracting some attention to the amazing things Queen’s Medical students have been doing year in and year out.”
Frid said he felt very honoured to receive the award and was quick to point out that he had much support along the way: “None of the projects I was involved in were individual, so I feel very grateful to the QMed faculty and students, particularly my classmates in the Class of 2015, that helped those projects be successful.”
Milestones: A tribute to our tenacious 2015 graduating class.
“One hundred and eighty-five”. That was the answer to my question.
The question, that I’d posed somewhat naively to our intrepid assessment coordinator, Amanda Consack, was “how many assessments have the 2015 class undertaken during medical school?”
“Do you mean everything?” she asked.
“Yes. Everything”, I answered, not wanting to sound wimpy.
In her typical fashion, Amanda provided me not only with the precise answer, but also the following breakdown by type and curricular phase.
Seems like a big number. Given that our curriculum spans a total of about 140 weeks, this means our graduates have been examined, in one form or another, an average of more than once per week during their four-year program. Closer examination reveals a somewhat less ominous picture. The 32 RATS (Readiness Assessment Tests) and 41 Quizzes are very low stakes methods used by our Course Directors to review course material. They’re therefore used as Formative assessments (to guide student learning), as are the 19 mid-terms utilized by most of our first and second year courses. The 37 “Assignments” and 7 mini-Scholar exercises, also have a largely formative role, and used primarily for teaching the Integrated Roles (non-Medical Expert Competencies). The major sources of Summative Assessment (for the purpose of ensuring students have achieved curricular objectives) are the end of course finals, OSCEs, NBME examinations and ITERs (In-Training evaluation reports) used in the Clerkship. Most of this apparently burdensome assessment load, therefore, is for the purpose of informing our students and curricular leaders as to the effectiveness of the learning process.
Nonetheless, 185 is a big number, and one might reasonably ask, “Why?” Is all this worth the considerable effort, expense (on our part), and stress (on the part of our students)? Do all these assessments make for a better-educated, more practice-ready graduate?
On one side of this issue is the “assessment drives learning” camp, which feels quite strongly that students must be led to the appropriate learning by their intrinsic desire/need/compulsion to succeed in examinations. This approach actually takes advantage of the drive for objective and external validation that allowed our students to succeed in the highly competitive medical school application process.
However, many feel that this approach promotes purpose-driven learning only and fails to establish the “deep learning” necessary for career success. It also falls short of embedding lifelong learning skills that are so important to ensuring ongoing physician competence. This camp, which has both strong faculty and student support, would much prefer a somewhat more Utopian educational environment where faculty directs learning and trust that students will enthusiastically undertake their studies because it’s fundamentally important to their own educational goals, personal interests, and future needs as physicians. They are, after all, adult learners selected from the most motivated and academically accomplished young people our society produces. These folks will go on to argue that postgraduate programs, the Medical Council of Canada and provincial regulatory bodies will, down the road, provide more than sufficient opportunities to ensure they are qualified to do fill the roles they choose to engage.
Unfortunately, this controversy can’t be resolved on the basis of the sort of objective, controlled evidence we all crave. There are no randomly controlled trials, and we would prefer not to assess only half our class and wait a generation to evaluate the fallout.
But that’s not to say we lack evidence of student success.
The Medical Council of Canada, National Board of Medical Examiners, and USMLE all provide external examinations that our students undertake and which allow comparison to other Canadian or North American schools, and show that our students do quite well in comparison to their peers. The Canadian Graduation Survey provides our school feedback from our graduates as to their satisfaction and sense of preparedness for residency, and shows that our students rank their educational experience at or above Canadian means. Perhaps most importantly, our students do very well in the increasingly competitive postgraduate matching process, with the 2015’s matching their 2014 predecessors in achieving a 100% match, with most achieving their first choices. The more informal feedback we receive regularly from preceptors who encounter our students and graduates in electives and postgraduate programs is very favourable and leaves our curricular leadership with the sense that “we must be doing something right”.
With regard to my view on the assessment controversy described above, I’ve come to feel that the purpose rather than the number of assessments is the key issue. In doing so, I’ve been drawn to the image of the Inukshuk. These are the elaborate arrangements of stones that have come to symbolize Inuit culture and are featured on the flag of Nunavut. I’d always assumed these were simply a native art form. I came to learn a few years ago from a prominent educator who I admire greatly that these are, in fact, cairns, left along migration routes in order to guide travellers to their destinations. In bleak and featureless landscapes, they point to the next step along the way. They also mark places
where provisions might be stored for the journey. They are, therefore, sources of both direction and nourishment. Our examinations, small and large, should do the same. They should provide encouragement to sustain the journey, and direct further learning.
And so, our 2015 graduates have truly encountered a series of milestones set along the way by a faculty that’s gone ahead, knows the trail, and is now intent on ensuring that those who follow will safely reach their destination. To our graduating students I would say that you have earned the right to be confident – confident in your knowledge, confident in your abilities, confident that your judgment and personal qualities will equip you for the challenges ahead. You are capable of being excellent physicians and, more importantly, excellent contributors to our profession and to our society. On behalf of all your proud faculty at Queen’s, I bid you safe travels.
Anthony J. Sanfilippo, MD, FRCP(C)
Undergraduate Medical Education
Student attention in class: Whose responsibility?
I just received a posting from Faculty Focus with the engaging title: Why can’t students just pay attention? Dr. Chris Hakala, the author, gives a really good overview of the dilemma many of us face when teaching: students are not engaged, are multi-tasking at best, and distracted at worst, and are not learning or retaining key concepts. How much responsibility should teachers bear for this lack of attention, and if we assume responsibility, what can we do about it? http://www.facultyfocus.com/articles/effective-teaching-strategies/why-cant-students-just-pay-attention/
Dr. Hakala defines attention, from the cognitive literature, as the idea that students have a finite amount of cognitive resources available at any given moment to devote to a particular stimuli from their sensory environment. To that end students’ attention is constantly shuttling between what they are experiencing externally and internally…At any given moment, [they] select from a large number of potential stimuli and focus on a small number of them. If class is interesting and there is activity, students can focus on those activities and work to remember that information for later use. However, when class isn’t engaging, students will find other things to occupy their attention.
You may recall reading my thoughts and others’ on multitasking and how it’s not really effective tasking at all. (http://meds.queensu.ca/blog/undergraduate/?p=822 and http://meds.queensu.ca/blog/undergraduate/?p=113
However, it’s a very challenging mind-set to change, as students appear convinced that they can multitask (they can) and learn well (a much more difficult proposition).
Dr. Hakala claims, as do many educators, that we as teachers should accept some of the responsibility for engaging students, and allowing them to focus on our teaching. He suggests the following:
- Ask questions and require students to write responses. Then ask again and have them read their answers to the class (not all, obviously, but a sampling).
- Have students respond to questions about readings or a previous class activity and bring those answers to foster peer discussions.
- Craft mini-lectures to include time for student comment, feedback, and response.
- Focus learning on student perspectives.
- Create rapport with students and build a classroom climate where students feel comfortable sharing their ideas.
I would add,
- Use small group learning, especially when all groups report to the whole class (this can be done by having the group answer a challenging multiple choice question that is the focus of the group task)
- Ask students to answer a question or solve a problem and share the answer with a peer. (See Peer Instruction by Eric Mazur.) Peer answers can be shared with the whole class. (Think, Pair, Share).
- Break up a lecture into 15 minute “chunks” punctuated by student activity.
- Create an outline, follow it, and demonstrate to students where in the activities of the outline you are.
- Use quizzes, or Readiness Assessment Tests to determine understanding
AND (and I’m going to be radical here) ask students to close their devices at certain points in the class when they’re not needed for taking notes, looking up references, etc.
Dr. Hakala states that there is evidence to support that deep processing, which happens when students are engaged (with only one task!), leads to better learning and cites Brown, Roedigger & McDaniel, 2014; Benassi, Overson, & Hakala, 2013. I highly recommend the Brown, Roedigger and McDaniel text: Make it stick: the science of successful learning. Cambridge, MA: Belknap Press. I’ve directed students and faculty to Prof. Roediggers’ writing before–he makes learning about learning accessible.
You can find a description of Benassi, Overson & Hakala’s book, Applying Science of Learning in Education: Infusing Psychological Science into the Curriculum (2014) at http://teachpsych.org/ebooks/asle2014/index.php.
Back to the blog article:
My stance is that teachers bear a some responsibility for student engagement…Yes, I know we all learned from 4 hour lectures; yes, we all had boring professors… and yes, look at how well we learned. However, we understand a lot more about learning now, and we have a different group of learners now. In the interests of good pedagogy as well as good role modelling and personal satisfaction, it’s important to set the stage so that learners will learn well.
What happens after that, however, is a student’s own responsibility. Part of the challenge in medical school comes from students not always being able to distinguish what is important at early stages. But as adult learners in medicine, and with patient safety, professionalism, and their future sound practice at stake, it’s important that we challenge students to take on part of the responsibility for learning, put the distractors away, and focus. It’s a hard habit to break for some of our “wired generation” but I believe it is worthwhile.
What are your thoughts on student and teacher responsibilities for engagement? And what ideas do you have to stimulate learning in class?
Towards higher ideals…Reflections on our current and our first accreditation experience
The first accreditation visit to the Queen’s School of Medicine occurred in October of 1909, and didn’t go particularly well.
The reviewer was Abraham Flexner, a rather determined iconoclast and career educator who had been commissioned by the Carnegie Foundation to carry out a review of all North American medical schools. Flexner undertook his charge with a shrewd earnestness that his biographers would later describe as “determination bordering on espionage”. He was both relentless and scrupulous, leaving no stone unturned and taking nothing at face value. He wasn’t above, for example, sneaking back into a school at night after the formal visit had ended to bribe his way into laboratories or wards from which he’d been tactfully steered away during the official tour. Some of his reviews were truly scathing, and would certainly lead to lawsuits today. Of the 155 American and Canadian medical colleges in existence at the time, fully 95 closed within the 20 years following the publication of his report in 1911.
Queen’s, at that time, was one of 8 Canadian schools in existence. Three were in Ontario (Toronto and Western being the others), three in Quebec (McGill and Laval, which had campuses in Montreal and Quebec City), Winnipeg and Halifax Medical Colleges. All were “proprietary” or commercial schools, operated by the medical community who charged students fees for instruction without standards for admissions, qualification of faculty, curriculum, teaching, assessment or any of the issues we take for granted today.
In 1909, the population of Kingston was about 20,000. In the School of Medicine 38 faculty (including 16 Professors) were responsible for the teaching of 208 students drawn largely from Ontario. The five-year program was expected to graduate students able to “comply with the requirements of the province in which they expect to practice”. The total income to the school, consisting entirely of student fees, was $19,978.
Flexner’s review was characteristically precise, perceptive and honest. He felt laboratory and library facilities were “adequate”. His major criticisms related to the paucity of clinical experiences – “the opportunities for out patient work are slight”. He expanded:
“The future of Queen’s is at least doubtful. It could certainly maintain a two year school; for the Kingston General Hospital would afford pathological and clinical material amply sufficient up to that point. But the clinical years require much more than the town now supplies. Its location – halfway between Montreal and Toronto, on an inconvenient branch line – greatly aggravates the difficulties due to the smallness of the community”.
However, he concluded by describing the school as “a distinct effort towards higher ideals”, and compared it favourably to a number of American schools he’d encountered in similar circumstances, holding out hope with the following observations:
- “liberal policy has largely overcome the disadvantages of location in a small town”
- “the thoroughness and continuity with which the cases can be used to train the student in the technique of modern methods go far to offset defects due to limitations in their number and variety”
The leadership of the school at the time was not amused. Dean Cowell’s report to the Board following the publication of Flexner’s report went as follows:
“The report of the Carnegie Foundation relating to Medical Education, published last summer, contained some statements and criticisms which are unfavourable to our school. As these were manifestly based on inadequate knowledge of the actual conditions, they have not been taken very seriously by the Faculty.”
(from Medicine at Queen’s 1854-1920, A.A. Travill).
And that, as they say, was that.
Last week, 105 years after the publication of Flexner’s findings, a team of six visited Kingston and again undertook to review our school, this time utilizing a rather extensive set of standards developed by both Canadian and American accrediting agencies. Although the full report won’t be available for a couple of months, the panel provided a preliminary report that appeared to find little fault with our core educational program, but did contain some rather eerie echoes of Flexner’s observations over a century ago. They felt the caseload available for teaching in some disciplines, for example, is low and the efforts put in place to offset this may require “ongoing monitoring”. At a number of points during the many meetings that occurred over the four day visit, the panel seemed to probe our ability, as a small school, to provide what appeared, almost surprisingly, to be a strong education and learning experience to our students.
The reality is that Kingston and the southeastern Ontario region does have a small and more senior population, tightly bound by our neighbouring schools, much larger and more diverse population centres, and our national border. This is as true today as it was in Flexner’s era.
So how does it work? Why has Flexner’s prediction, honestly and pragmatically derived, not proven true? How is it that this “effort to higher ideals” has not only survived but, I would humbly suggest based on our recent review, flourished despite the geographic and epidemiologic challenges?
The answer to these questions was in evidence this past week when our Dean, Vice-Deans, Associate Deans, Department Heads (every one of them by the way), hospital partners, Program Directors, Course Directors, Competency Leads, Administrative staff and students, met in a series of meetings with the accreditation team. What was apparent to myself and to the review panel as each group came through, was the same spirit of determined commitment to medical education that has sustained our school through the generations since Mr. Flexner’s visit. Such commitment trumps issues of size and location, converts potential liabilities into advantages and results in development of a particular and enviable learning community where education is valued in not only theoretical but also highly practical ways, where students are vitally involved, where innovation is very much encouraged, where conventional thinking is challenged, where problems can be solved with a phone call today rather than a meeting next month, where effort required is no barrier to a good idea, where people know, respect and support each other and share in the common mission because, quite simply, they care. They care about providing the best possible medical education experience for our students, they care about our school, and they care about each other. It’s really that simple, and it’s no secret. An “effort toward higher ideals”, indeed.
Anthony J. Sanfilippo, MD, FRCP(C)
Undergraduate Medical Education
A.A. Travill. Medicine at Queen’s 1854-1920: A peculiarly happy relationship. The Hannah Institute for the History of Medicine. 1988.
Abraham Flexner. Medical Education in the United States and Canada. New York. The Carnegie Foundation for the Advancement of Teaching. 1910.
Thomas Neville Bonner. Iconoclast: Abraham Flexner and a Life in Learning. Johns Hopkins University Press. 2002.