Figuring out what’s important in a faculty evaluation report

Post Thumbnail

Critics only make you stronger. You have to look at what they are saying as feedback. Sometimes the feedback helps, and other times, it’s just noise that can be a distraction.

Robert Kiyosaki

 

Separating the useful feedback from the noise in students’ comments on faculty evaluation questionnaires is an annual challenge for all university instructors– not just at Queen’s School of Medicine. It was recently the topic of a Faculty Focus article by Isis Artze-Vega, associate director of the Center for the Advancement of Teaching at Florida International University. She offered solid advice for those of us who feel angst over student evaluations.

As a side-note: the headline on the Faculty Focus article points to “cruel student comments” but just because there’s negative feedback doesn’t mean the comments are (or should be) cruel or rude. Student in the School of Medicine are given guidance about completing course and faculty evaluations at the beginning of every year. They’re encouraged to be professional in completing them, so while there may be constructive feedback, it should never degenerate into merely cruel criticism. Students are also encouraged to provide concrete suggestions for ways to improve.

Dr. Artze-Vega suggests seven key approaches to responding to student evaluations.

First, she advises faculty to analyze the data. The Education Team routinely does this for course evaluations for course directors and the Course and Faculty Review Committee, but not for individual faculty evaluations. Analyzing comments is a great starting point; otherwise, human nature often has people hyper-focusing on the wrong things. Are you reading an outlier opinion, or is there a theme in multiple students’ comments? “Identifying themes will help you determine whether they warrant a response,” Artze-Vega writes.

Ask any actor or director and they’ll tell you: negative feedback is easy to remember. American film director Peter Farrelly has said: “With all of my films, if I get one bad review and a bunch of good reviews the bad one is the only one that will stay with me.” Artze-Vega cautions to resist the lure of the negative. Don’t automatically dismiss a negative comment, but “consider: Am I focusing on this because it’s ‘louder,’ or because it’s a legitimate concern?”

Considering feedback this way flows into Artze-Vega’s third key: Let your critics be your gurus. Citing a New York Times article, she points out that “we often brood over negative comments because we suspect they may contain an element of truth.”

A fourth approach is to find counter-evidence to negative comments. You can look for or remember comments that contradict the negative one. (If your faculty evaluation report is anything like some course evaluation reports, sometimes, you’ll find these comments in the same evaluation report from other students).

Artze-Vega stresses that “we should devote at least as much time to students’ positive comments as their negative ones”, so her fifth key is dwell on the positive ones. If you hyperfocus on negative feedback, you can lose sight of the many things you are likely doing well – and that students appreciate. To aid in this, she further advises to read them with a friend. “A more objective party can help you make sense of or notice the absurdity of the comments because they’re not a personally invested in them.”

Finally, Artze-Vega advises to be proactive. “If you don’t conduct this analysis yourself, you’ll be at the mercy of whomever is charged with your evaluation—and they probably won’t be as thorough,” she points out. “Also, take the time to provide explanations about any off-the-wall student complaints, so that your reviewers don’t draw their own conclusions.”

One way to be proactive, is to solicit feedback earlier, when you can still make adjustments for this cohort of students, rather than waiting for the end-of-semester one to help with next year’s planning.

When she’s teaching a full course, Sheila Pinchin, Manager, Educational Development and Faculty Support has always used her own evaluation forms three weeks into a course and three weeks later to get just this type of feedback.

Terry McGlynn, an associate professor at Cal State, also advocates this approach as one of his tricks-of-the-trade to avoid bad teaching:Young Woman Sitting Looking at Laptop Screen

“I often use a supplemental evaluation form at the end of the term. There are two competing functions of the evaluation. The first is to give you feedback for course improvement, and the second is to assess performance. What the students might think is constructive feedback might be seen as a negative critique by those not in the classroom. It’s in our interest to separate those two functions onto separate pieces of paper. Before we went digital, I used to hold up the university form and say: “This form [holding up the scantron] is being used by the school as a referendum on my continued employment. I won’t be able to access these forms until after the next semester already starts, so they won’t help me out that much.” Then I held up another piece of paper [an evaluation I wrote with specific questions about the course] and said, “This one is constructive feedback about what you liked and didn’t like about the course. If you have criticisms of the course that you want me to see, but don’t think that my bosses need to see them, then this is the place to do it. Note that this form has specific questions about our readings, homework, tests and lessons. I’m just collecting these for myself, and I’d prefer if you don’t put your names on them.” I find that students are far more likely to evaluate my teaching in broad strokes in the university form when I use this approach, and there are fewer little nitpicky negative comments.”

 

If you do decide to use this type of mid-course feedback, keep your questions few and focused. (Otherwise, there’s potential for evaluation-fatigue on the students’ part, which lowers the quality of the very feedback you want).

 

If you teach in the Queen’s UGME program and would like some assistance in separating the useful feedback from the noise, I’m available to assist you with this. Drop me an email. Reach me at theresa.suart@queensu.ca

See more on this topic from an earlier UGME blog post here.

 

 

 

Leave a Reply

Post Timeline

Curriculum Committee Information – September 28, 2017
Published Wed, November 15, 2017

Faculty and staff interested in attending Curriculum Committee meetings should contact the Committee Secretary, Candace Miller (umecc@queensu.ca), for information relating to agenda items and meeting schedules. A meeting of the Curriculum Committee was held on September 28, 2017.  To review the topics discussed at this meeting, please click HERE to view the agenda. Faculty interested in reviewing the minutes of the September … Continue reading

Grade Inflation – the “dirty little secret” of academia
Published Mon, November 13, 2017

“Would any of us have gotten into medical school today?” This was the tongue-in-cheek question I posed to my classmates at our medical school reunion last year. They were rather amused by it and, being very much aware of the high academic standards required by our current admissions processes, believed the answer was an obvious “no”. I tried to raise … Continue reading

Facebook thinks I’m a doctor…
Published Mon, November 6, 2017

  And other unusual things that happen when you’re an educational developer at a medical school It’s a unique and interesting thing being one of the non-medically-trained employees who work (mostly behind the scenes) to help run the undergraduate medical education program at Queen’s. On the one hand, friends and family can sometimes think I’ve magically completed medical school in … Continue reading

Nominations open for next Exceptional Healer Award
Published Mon, October 30, 2017

Instilling the values of patient-centered care is one of our goals in the UGME program. It’s also what the Kingston Health Sciences Centre Exceptional Healer Award recognizes in physicians from both the Hotel Dieu and KGH sites. Launched earlier this year, the Exceptional Healer Award is sponsored by the KHSC Patient & Family Advisory Council. It honours a physician who … Continue reading

Students striving to make a difference in our community
Published Mon, October 23, 2017

One of the attributes that our Admissions Committee works very hard to identify in applicants is a commitment to service. This has multiple dimensions, involving service to both individual patients and communities. It’s therefore always very gratifying to learn of efforts such as that described below in todays guest article provided by students Lauren Wilson, Katherine Rabicki and Melissa Lorenzo. … Continue reading