Using I.C.E. to build objectives and activities

It’s snowy and icy out–a perfect time to learn to use I.C.E. to build your learning objectives and activities for your courses and individual sessions. The I.C.E. model stands for “Ideas, Connections and Extensions,” and was developed by Dr. Sue Fostaty-Young and Dr. Bob Wilson here at Queen’s.  When you’re planning a session or a course, use the concepts of building from Ideas (or facts and recall), to Connections (higher order thinking processes of analysis and application) and to Extensions (even higher order thinking processes of evaluation and creation).  These will help you design activities that lend themselves to different levels of thinking and doing.

The I.C.E. Model:

Ideas Connections Extensions
Knowing about

 

Remember/Understand

 

  • Factual recall of basic information
  • Grasp of elemental concepts
  • (e.g. conventions, principles, procedures, trends, laws)

 

 Understanding how and why”

 

Analyze/Apply

 

  • Recognizing general ideas across different contexts
  • Demonstrating relationships and connections among concepts
  • Connecting prior knowledge and experience

 

“Thinking Beyond”

 

Evaluate/Create

 

  • Predicting future outcomes
  • Proposing solutions
  • Justifying a position
  • Evaluating outcomes
  • Designing or building something new
  • Changing contexts

 

 And, after planning the types of learning activities, here are some helpful verbs that will assist you in determining the learning objectives:

Verbs for I.C.E.

Ideas Connections Extensions
Define,

describe,

explain,

label,

match,

identify,

list,

locate, recognize

 

Apply,

compare,

contrast,

classify,

organize,

categorize,

distinguish,

interpret,

integrate,

modify,

rate,

solve

 

Design,

develop,

diagnose,

evaluate,

extrapolate,

judge

predict

 

Adapted from Fostaty Young, S. & Wilson, R.J. (2000). Assessment and learning: The ICE approach. Winnipeg, MB: Portage and Main Press.

Erickson, L. B. & Strommer, W.D. (1991). Knowing, understanding and thinking: The goals of freshman instruction. In Teaching college freshman (pp.65-80). San Francisco: Jossey-Bass.

 

Posted on

What is the difference between outcomes and objectives?

You may have heard the terms objectives and outcomes used interchangeably and certainly some of the literature is confused upon the point.  Let’s try to clarify the distinction so that we can build our outcomes for our programs, and learning objectives for our courses and for sessions.

Outcomes Objectives
The essential and enduring knowledge, abilities (skills) and attitudes (values, dispositions) that constitute the integrated learning needed by a graduate of a course or program. Describe in detail the behaviours that students will be able to perform at the end of a unit such as a class, and the conditions and criteria which determine acceptable performance.
Achieved results or consequences; evidence that learning took place Intended Results
High Level, overarching Specific activities/assessments that lead to outcomes
Broader, larger scope, knowledge/skill Specific discrete units of knowledge/skill/competency
Accomplished over time in several learning experiences Can be accomplished in a short period of time—useful for a class session
Refer to reliable demonstrations of performance—results of a course/program—achieved results Can be statements of intent but should be linked to assessment of specific skills/knowledge and to specific strategies suitable for the objectives
Contain conditions under which the student performance will be assessed, as well as criteria for assessment Contain conditions under which the  specific student performance will be assessed, as well as criteria for assessment

Whether you are on a program or curriculum committee to develop outcomes, or are developing objectives for your course or individual session, here are helpful verbs to use and some “weasel verbs”  to avoid:

Words Open to Many Interpretations Words Open to Fewer Interpretations—behavioural and  measurable
To know to write; to explain
to understand to recall
to really understand to identify
to appreciate to sort, to organize, to compare
to fully appreciate to solve, to deconstruct, to follow a model or approach
to grasp the significance of to construct
to enjoy to evaluate
to believe to analyze
to be aware of to estimate

 

Sources:

Developing Clear Learning Outcomes and Objectives http://www.thelearningmanager.com/pubdownloads/developing_clear_learning_outcomes_and_objectives.pdf

How to Write Program Objectives/Outcomes http://www.assessment.uconn.edu/docs/HowToWriteObjectivesOutcomes.pdf

Developing Effective Learning Outcomes & Objectives http://www.cmduke.com/2011/07/23/developing-effective-learning-outcomes-objectives/

 

Posted on

Case Reports Database

Dr. Kanji Nakatsu shared this resource with us recently.  It’s a bank of Case Reports, from Biomed Central and supplemented by the Journal of Medical Case Reports.  It is searchable and freely accessible.  This is a resource for physicians, but may also be used in medical education.  “By bringing similar case reports together, through the Cases Database, researchers and clinicians can start to look for new knowledge – new associations, new side effects, new thoughts about disease processes, new understandings about the impact of disease on our patients and our communities.”

Access it by going to

Posted on

Revised Policy: Student Initiated Extracurricular Learning Activities

The Teaching and Learning Committee is requesting comments on revisions made to the  policy on Student Initiated Extracurricular Learning Activities.  A summary of the changes are listed below. The full revised policy may be viewed by clicking here.   Please submit all comments no later than March 8, 2013 by using the “Discussion” Comments: Policy on Student Initiated Extracurricular Activities

Student Initiated Extracurricular Learning Activities have been in place for many years.  A policy is needed only to ensure that these activities complement but do not supersede planned curricular activities and to direct those activities that become part of the “Dean’s Letter”. The last version of this policy, 2007, has received only a few changes:

  • A new definition and background statement have been added specifying that these activities must not impinge on curricular time.
  • The new Student Liaison position in the Aesculapian Society which offers substantial assistance to students, UGME Office and Teaching and Learning Committee, has been woven into the procedures.
  • The procedures have been separated from the policy.
  • The Teaching and Learning Committee now assumes responsibility for vetting and approving these activities as part of their mandate to oversee opportunities for independent and lifelong learning.
  • The students’ submission of attendance records only will trigger the entry into Dean’s Letter.

Posted on

Medical School Admissions: Striving for fairness despite “ill-designed” tools

February is, easily, the most difficult month of the year for many involved in undergraduate medical education, including Deans, admissions committees and administrative staff.  This is not simply because of the long and dreary Canadian winter.  It’s during this month that letters go out to applicants for admission indicating whether they’ve advanced to the next stage of the process: the on-site interview.  All those involved in the selection process struggle with the knowledge that, for every letter that brings welcome relief and encouragement, several will result in bewildered disappointment.  Two brief applicant profiles may serve to illustrate the issue.

Jessica is a bright, articulate and engaging young woman who, for as long as she or anyone can remember, has wanted to be a Doctor.  She graduated from high school at the top of her class, with numerous awards recognizing not only her academic accomplishments, but also student leadership and community involvement.  She received multiple university entrance scholarships and undertook an undergraduate program with courses that would provide a basis in biologic and physical science, which she feels is relevant to the study of medicine, but also selected to optimize her marks.  She is very successful, maintaining a 3.8 GPA over her first three years.  She also undertook a variety of volunteer activities, locally and abroad, involving health care in various settings.  She took the Medical College Admission Test (MCAT) after both studying from a manual and taking a preparation course at significant expense.  She did generally well, but was concerned about her mark in one of the four exam categories.  Jessica applied to our medical school, but failed to even get an interview.  This was related entirely to the MCAT score, as she feared.

Matt is not only an excellent student finishing in the top 5% of his high school graduating class, but also an elite athlete who accepts a full scholarship to an Ivy League university.  He chooses this school because it will allow him to pursue his interests in philosophy and political studies at an institution with an international reputation for excellence in both disciplines.  While there, he continues to excel academically, while becoming an accomplished varsity athlete.  He also develops an interest in Medicine and, specifically, Public Health.  He decides to apply to medical school and takes the MCAT, in which he excels in all categories.  He would like to return to Canada for medical school, but also fails to even get an offer for an interview, largely because the grades for his philosophy and political science courses, although near the top of the class for every course, fall below our GPA cutoffs.

Jessica, Matt, their families, and everyone who knows them and their career aspirations, are understandably devastated and rather perplexed.  “How can this be?”

Although these are both fictional accounts, a recent review of our applications at Queen’s shows that no fewer than 247 submissions matched the “Jessica” scenario almost exactly.  The number of “Matts” is more difficult to determine, but likely similar and probably underestimated because many people in such circumstances will decline to even apply, recognizing the GPA issue.

For every jubilant success, we know there are about 7 “Jessicas” and “Matts” who will be very disappointed and may have to set aside or delay their life’s dream, despite being very capable, motivated and deserving.  That reality is also personally distressing to the faculty and staff involved in the admissions process who, recognizing they cannot admit every applicant, endeavor diligently to develop fair and equitable processes.

A few realities about the medical admission process in Canada:

Among Canadians, there is a very high demand for medical education.  At Queen’s we received 3818 applications for our 100 positions this past year. All Canadian schools receive many times more applications than they can accommodate. Statistics collected and published annually by the Association of Faculties of Medicine of Canada indicate that the Canadian schools collectively received 34,048 applications for their 2,877 total available positions in 2011.  Assuming an application per candidate ratio of 3.3 (as Ontario statistics would suggest), it would appear that at least 10,318 individuals submitted applications that year.  The hunger for a career in medicine is such that increasing numbers of Canadians are enrolling in medical schools in Australia, the Caribbean, Ireland and other countries, at considerable personal expense and with no assurance of postgraduate training or eventual qualification in Canada.  Although no accurate data is available, it’s estimated that there are now more Canadians studying Medicine outside Canada than within.

Applicants to Canadian medical schools are knowledgeable regarding the process, and highly accomplished academically.  Although, again, no data is collected on this subject our observation at Queen’s, which seems to be shared by other schools, is that the average GPA, MCAT scores and personal experiences reported by our applicants are increasing each year.  Applicants understand the “system” and are highly strategic as they undertake their education and personal activities.

The number of medical school positions in Canada is fixed by public authority.  Medical education is expensive and largely subsidized by provincial governments.  Those governments therefore define the number of available positions, based loosely on anticipated demands for physicians.  These estimations have fluctuated in the past such that we have seen periods of both contraction and expansion.  At present, there are no plans in Ontario for expansion.

Medical schools place a priority on fairness and equity in their application processes.  In the face of the virtual impossibility of selecting the “most worthy” from so many worthy applicants, schools opt to ensure objectivity and fairness in their processes.  They are therefore drawn to metrics that provide some basis for objectivity.  Unfortunately, all available metrics are inherently blunt and imperfectly aligned with the qualities all would agree are important.

Winston Churchill could have been talking about medical admissions when he famously described golf as “a game whose aim is to hit a small ball into a small hole, with weapons singularly ill-designed for the purpose”.   Academic records, the MCAT, and quantified assessment of reported personal experiences all have significant shortcomings, as our examples above illustrate, but have the significant advantage of providing a numerical assessment by which candidates can be ranked without prejudice.  Panel interviews and mini-medical interviews (MMIs) are being used increasingly by medical schools to better assess applicants personal qualities, and are certainly an improvement, but are very resource-intensive and difficult to conduct and evaluate in a reliable manner.  It’s therefore not possible to apply such methods to the large number of applicants.  Hence the staged application process and reliance on other academic and test metrics.

So, one must ask, do we have a problem?  Despite all these shortcomings, the students who are finally admitted to our medical schools are an exceptional group of very talented, intelligent and capable young people who, with rare exceptions, have all the necessary qualities to become outstanding physicians.  Our processes, although inherently blunt and likely misaligned, are objective and scrupulously fair to all applicants.  Importantly, the Jessicas and Matts of the world, and their families, can perhaps take some small comfort in the knowledge that they are far from alone and have been treated fairly.  Furthermore, medical schools recognize that even if they could personally interview or meticulously assess every applicant, most would still be disappointed.  So, should we change and, if so, how?  I welcome viewpoints, and will make that issue the subject of the next blog.

Anthony J. Sanfilippo, MD, FRCP(C)
Associate Dean,
Undergraduate Medical Education

Posted on

Updated Faculty Resources Community Available

briefcaseThe newly-updated Faculty Resources Community is now available in MEdTech Central. This online resource contains great teaching and assessment ideas, highlights of Curriculum Committee, notes and slides from the retreats, and more.

The resource material available includes refresher instructions on the audio-visual equipment in teaching theatres 132 and 032 (including a map of the numbered student microphones), e-learning resources and links to the small group learning community.

This Faculty Resource Community is open to all faculty at the School of Medicine.  For more information, please contact Sheila Pinchin (sheila.pinchin@queensu.ca) or Theresa Suart (theresa.suart@queensu.ca).

Posted on

Family Medicine and the Hidden Curriculum

Lessons from Medical Variety Night

Last November’s Medical Variety Night provided an impressive peek into the musical and comedic talents of our students.  Andrea Winthrop, Steve Archer and I were asked to serve as a “Judges Panel” to determine the best class skit.  We were all amazed at the poise and creativity on display.

The evening also provided a few lessons for both faculty and students as to how well intentioned humour and satire can appear quite different before a mixed audience not familiar with the contexts employed.  A number of conversations have ensued which I believe have been illuminating and instructive for both students and faculty.  As in the patient care context, “near misses” are opportunities to learn and avert more serious problems in the future, and I believe we have, as a school, availed ourselves of the opportunities this year’s production provided.

A theme that I and others in attendance found particularly troubling related to a number of references to Family Medicine as a less-than-appealing career option.  This perception is hard to fathom given that we a have superb and award winning faculty teaching Family Medicine.  Our Family Medicine training program is widely recognized as one of the best and most sought after programs in the country.  In addition, Family Medicine is, arguably, the most demanding of specialties.  In open and frank discussions with a number of students on this issue, a few underlying causes came to light which I found illuminating and felt would be useful to share with faculty.  They provide superb examples of the “Hidden Curriculum”, a term used to describe unintended influences that affect student learning, and are known to be very powerful shapers of student attitude and behavior.  So, in no particular order:

  1. Engaging Uncertainty.  Students find Family Physicians they encounter to more commonly express uncertainly in their ability to resolve patient presentations.  This is in contrast to other specialists who they find characteristically more definitive in their approach.  Family Physicians more commonly use statements like “we’ll have to look it up”, or “don’t be afraid to say you don’t know”.   With respect to other specialists, the expression “seldom wrong but never uncertain” comes to mind (my quote, not theirs).  Importantly, students do not see this difference as inappropriate or as reflecting any lack of competence, nor are they so naive as to believe other specialists always have the answer.  In fact, the students I met uniformly expressed admiration and respect for Family Physicians they encountered, and their ability to manage a diversity of patient populations and presentations.  However, it’s clear that our students are accustomed to success, and many are not yet comfortable facing uncertainty in their lives, or in their future practices.
  2. Technical/procedural expertise.  Many of our students are technically very savvy and excited by the prospect of being on the “leading edge” of innovation and application of emerging technologies and procedural approaches to various conditions.  Simply stated, they like the “toys” of modern medicine and they’re excited about applying evolving procedures.  They perceive that these exciting new approaches as the exclusive domain of sub-specialists.
  3. Prestige”.  Hospital in-patient services remain at the core of clinical training for our students.  Properly run and supervised, they are superb environments in which the learning of pre-clerkship can be applied to real patients, with appropriate overlays of scholarship, professionalism, advocacy, collaboration and all the intrinsic competencies we have adopted within our curriculum.  Although formal teaching remains valuable during these rotations, we all recognize that the major component of learning occurs through active participation as part of the team, and by observation of “real life medicine”.  With respect to Family Medicine, these rotations are problematic in two important ways.  Firstly, virtually no Family Physicians are involved or even visible during these rotations.  Secondly, and most disturbingly, they often see instances where primary care and primary care providers are disparaged.  A casual reference suggesting that a patient was inadequately cared for prior to admission, or a concern that appropriate care will not be continued after discharge can, in a stroke, undo all prior teaching.  These observations in the clinical setting trump teaching in the pre-clerkship.  Our actions, it would seem, speak more loudly than our words.
  4. Money.  I have become convinced that the single most powerful expression of Hidden Curriculum in our society is the OHIP Fee Schedule.  Students are very aware of the differential reimbursement of physician groups, and the high premium paid for procedural work relative to patient assessments.  This of course, results in two hugely damaging consequences, the equation of financial with professional “value” or “prestige”, and the enticing allure of higher income to students facing increasing debt loads by graduation.

So what can be done?  A few suggestions, humbly submitted for consideration:

  1. Awareness of these influences, and of the Hidden Curriculum in general.  Hopefully this article is a start.  I hope it will generate some discussion, particularly at department meetings.  Dr. Leslie Flynn is chairing a group of which I’m a member to study and address Hidden Curriculum issues, and I think this may provide some focus for those discussions.  This awareness must extend to physicians of all disciplines who teach and supervise our students, particularly in the practice setting.  Those who attend on these services are, in my experience, largely unaware of the serious impact of casual commentary, and almost never intend to disparage any other specialty.
  2. Within our curriculum, developing strategies to address the “uncertainty principle” in a more open fashion.  This is both an academic and student wellness issue.  Our students require means to cope with the uncertainty that will inevitably develop in their professional and personal lives.
  3. Serious consideration of the troubling question:  To what extent do our admission processes pre-determine career choice?  Medical school admission remains a highly competitive process (applicant to admission ratio 38:1 at our school), and is likely to become even more competitive in the near future.  This environment favours the goal oriented, determined self-starter who is able to engage this single goal with appropriate compromises and sacrifices along the way.  It can be argued that such “survivors” will be naturally attracted to practice environments that provide definitive resolution of problems, technical mastery and perceived prestige.  There is a recognition, even embedded in the Future of Medical Education in Canada initiative, that admission processes should favour resiliency, personal maturity and problem solving, qualities valuable to any physician and not necessarily reflected by academic success.  Our admissions committee has, in fact, been inoculating these considerations into their procedures for the past few years.  However, as for all schools, academic success remains a key component of the application process.  Perhaps it’s time to consider more radical approaches.
  4. Increasing Family Physician presence in the hospital.  Our students perceive that in-hospital care, and the acuity, complexity and technologic innovation that goes with it, is the exclusive domain of sub-specialists, and fail to appreciate the role of Family Physicians in the continuum of care.  They also get little exposure to the in-patient care provided by Family Physicians in smaller communities.  Our Integrated Clerkship and “Week in the Country” programs address this to some extent, but we need to develop and engage initiatives to integrate Family Physicians effectively into the care of our in-patients.
  5. Advice regarding financial planning and practice management.  Although we can’t influence the fee schedule, we can certainly provide our students with sound financial advice to lessen any economic drivers of career choice.

I would like to end this article by thanking the many students who were willing to speak to me candidly about this issue.  I welcome their further commentary and impressions of faculty.  Open discussion is always the first and perhaps most necessary step to improvement.

Posted on