The Royal College of Physicians and Surgeons of Canada (RCPSC) and the College of Family Physicians of Canada (CFPC) conduct regular on-site accreditation visits to ensure the quality of the residency programs at Queen's University. In order to prepare for these visits and to identify strengths and areas for improvement, an internal review process has been established.
Internal reviews are an integral component of the accreditation process at Queen's University and are conducted at least two years prior to the regular RCPSC/CFPC visits. These internal reviews are intended as a mechanism to assist Queen's in maintaining the quality of its residency programs. They also provide the Postgraduate Medical Education Committee (PGMEC) and Program Directors with valuable information about their programs and enable them to take corrective measures to address any weaknesses before the next RCPSC/CFPC survey.
The objectives of the Internal Review are as follows:
- To assess the strengths and weaknesses of each program and to give programs a chance to improve prior to the next Accreditation visit
- To consider and evaluate all residency education sites, including elective experiences
- To provide a means for ongoing quality improvement of each residency program
An agenda template will be provided to the program by the PGME Office. The Program will complete and return this agenda to the PGME Office. Each program must also make the logistical arrangements for its own review. This includes a room booking, morning refreshments, and lunch. Costs are the responsibility of the program.
In preparation for the Internal Review, each Program Director must complete the RCPSC/CFPC Pre-Survey questionnaires. These surveys assist the Program and the Review Team evaluate whether the standards of accreditation are being met. After completion, the Pre-Survey documentation must be submitted to the PGME Office on four (4) USB keys. PSQs may be obtained from the PGME Office; please contact Karen Spilchen, Program Coordinator, for further information.
The Chief/Senior Resident of the program being reviewed will be contacted by the PGME Office asking him/her to consult with residents in order to prepare a report of the program. This critique will be sent to the Associate Dean and will remain confidential, only being released to the Internal Review Team.
Each Internal Review Team will be comprised of the following membership:
- Resident Representative
- External Reviewer
The Chair shall be a Program Director from another discipline.
The Resident Representative shall be a resident from another discipline.
The External Reviewer shall be a Program Director from another centre in Ontario/Quebec who does not have a conflict of interest and has a good knowledge of the specialty being reviewed.
The Internal Review Team will be provided with the following information:
From the PGME Office prior to the meeting
- List of committee members
- RCPSC/CFPC Status Letter, Site Survey Report, and Faculty Response
- Pre-Survey Documentation (e.g., questionnaires, appendices, etc.)
- Specific Standards, Objectives of Training, and Specialty Training Requirements
- Resident Report
- General Information Concerning Accreditation of Residency Programs
- General Standards of Accreditation
- College checklist
- Template for Chair's Internal Review Report
From the Program on the morning of the meeting
- Residency Program Committee minutes for previous two years
- Examples of resident files
Each member of the Internal Review Team is required to review the provided documentation before the review. This should take approximately one to three hours and the purpose is to ensure that the program is meeting the standards of accreditation and to prepare questions addressing these standards.
A series of interviews must take place with the Program Director, teaching staff, residents, and with the Residency Program Committee. Visits to individual sites should take place as appropriate. All residency education sites and elective experiences should be reviewed by the Internal Review Team. There should be a careful assessment of the quality of the program based on the general and specific standards of accreditation as outlined by the respective College.
Meeting with the Program Director
The meeting with the Program Director is to ensure that the program structure meets the Accreditation Standards. For example:
- Does the program hold regular Residency Training Committee meetings and are minutes kept?
- Is there appropriate representation on the Committee (i.e., Faculty members, Educators, Teaching Site Representatives, elected Resident Representatives)?
- Does the Program have a formal Evaluation process? Are Evaluation forms completed in a timely manner and discussed with residents?
- Does the program provide opportunities for residents to evaluate the quality of their education?
- Are residents provided with Educational Objectives, a structured curriculum or is the Curriculum incident-driven?
- Are residents provided an opportunity for graded responsibility?
- Are exam Results consistent or is there a significant reduction in performance over the last few years? Are they consistently below national average? If so, this may be a sign of a problem in the educational program.
- How did the program address weaknesses identified during the last review?
- Have educational initiatives been developed or have they been updated since the last review?
- How does the program deal with residents in difficulty?
Meeting with the Department Head & Division Chair
This meeting serves to identify the involvement of the Head in the Postgraduate program.
Questions to consider include:
- Where does Postgraduate Education fall within the Department’s mandate?
- How much does the Head support the Program Director?
- Are adequate resources provided (administrative, financial, physical, material) by the Hospital?
Meeting with the Residents / Chief Residents
The reviewers will meet with all residents in the program. In large programs it will be customary to meet separately with the junior residents and senior residents.
This meeting is to allow the residents to identify the strengths and weaknesses of the program. It is important to be positive and give the residents a feeling of openness. It is suggested that the Review Team Chairperson start the discussion with an opening remark by indicating the purpose of the Internal Review and that residents should feel free to discuss any issue. It is also important to emphasize that all discussions will remain confidential and anonymous should there be a need to identify issues in the report.
Question the residents on the following points:
- Do they think the program is meeting the specialty training requirements?
- Are residents made aware of their educational objectives and are these being met?
- Do the senior residents feel that the program provides adequate opportunities to develop their skills necessary to function as a specialist physician?
- How do residents perceive the service to education balance? Do they feel there is a good balance between education and service expectations?
- Do residents feel there is open communication within the program?
- What are the strengths of the program?
In the event that issues of intimidation or harassment are raised, it is important to listen to comments, however it is also important to ensure that this issue does not drive the entire discussion.
Meetings with Research Director, Teaching Faculty, and Residency Program Committee
These meetings are to obtain feedback on how the members of the program perceive the quality of the education and the structure of the program. It is also an opportunity to question members of the Residency Program Committee. Questions to consider include:
- Is the Committee dealing with issues identified at meetings and is there a follow up mechanism?
It is recommended that a portion of this meeting should be with the Residency Program Committee without the Program Director, Division Chair, or Department Head present, to allow for frank discussion.
Preparation for Exit Report
This meeting is to allow the Internal Review Team to come to a consensus on the apparent strengths and weaknesses of the program and to prepare for the Exit meeting with the Program Director. The Internal Review Team should decide what they will report on during the Exit meeting.
The exit meeting is simply to outline to the Program Director the findings of the review and to obtain clarification on certain issues. Be sure to identify the strengths and weaknesses of the Program. Give an opportunity for the program to provide feedback on the comments you made. This will help the Internal Review Team to identify whether a specific issue is really a weakness/problem or if it was just a perception.
Prior to the review, a template will be distributed to the Chair to facilitate the preparation of a written report, which must include the strengths and weaknesses of the program and may include specific recommendations for continued development and improvements. This report must be submitted to the Associate Dean no later than three weeks following the review. Internal Review Reports will follow the templates from either the Blue Book (for RCPSC programs) or the Red Book (for CFPC programs). The Report will be sent to the Program Director and Department Head, who will be invited to an Internal Review Sub-committee meeting at which the Chair will present the report. This presentation will occur within two months of the review.
Internal review reports are deemed to be internal documents of the University. However, reports of the internal review of all programs are to be provided to the survey team chair prior to the regular RCPSC/CFPC survey to enable the chair to assess the efficacy of the internal review process. With the exception of those programs for which internal reviews have been mandated by the respective Accreditation Committee, internal review reports are not available for review by the surveyors at the time of a site visit and are not used by the College in making decisions regarding the accreditation status of individual programs.
The following costs are the responsibility of the Program:
- Refreshment breaks and lunches for Internal Review Team
- Travel expenses for External Reviewer if he/she is from outside of Ontario/Quebec. Contact Karen Spilchen to review funding.
The following costs are the responsibility of the Postgraduate Medical Education Office:
- A modest honorarium for the External Reviewer (currently $750)
- One night of accommodation for External Reviewer, unless the review encompasses a full day, in which case two nights of accommodation will be provided)
- Travel expenses for External Reviewer for travel within Ontario/Quebec
Six months prior to the Internal Review [Office/Person responsible]
- Agenda template distributed to Program Assistants [PGME]
- Pre-Survey Questionnaires distributed to Programs [PGME]
- Consult with potential External Reviewers [Program Director]
- Set date, ensuring availabilty of Department Head, Division Chair, Faculty, and Residents [Program Director]
Four months prior to the Internal Review
- External Reviewer contact information provided to PGME [Program Director]
- Preparations begun for internal review [Program]
- Department Head, Division Chair, Faculty and Residents provided with details of visit [Program Director]
One month prior to the Internal Review
- Pre-Survey Documentation due at the PGME Office [Program Director]
- Internal Review Agenda due at the PGME Office [Program]
- Resident Report due at the PGME Office [Chief / Senior Resident]
One month after the Internal Review
- Written Report due [Chair of the Internal Review]
- Report sent to Program Director and Department Head [PGME]
Report Presentation to Internal Review Sub-committee
- Chair to present report on a date coordinated by PGME [PGME]
Responsibilities - Program / Program Director
- Contact colleagues from Ontario/Quebec to solicit interest in becoming an External Reviewer
- Provide PGME Office with contact information for External Reviewer, including email address, mailing address, and phone number
- Inform all involved faculty and residents of duties and responsibilities for upcoming review
- Ensure availability of Department Head and Division Chair
- Book rooms for the review
- Arrange refreshment break and meal(s) for internal review team
- Complete and submit four USB keys containing the Pre-Survey Documentation to the PGME office
- Complete and submit the internal review agenda to the PGME office
- Make available the following documentation to the review team on the morning of the review:
- Residency Program Committee minutes
- Examples of Resident Files
Download the Program Review Checklist.
Responsibilites - Postgraduate Medical Education Office
- Contact External Reviewer to coordinate accommodation and travel arrangements
- Arrange honorarium for External Reviewer ($750)
- Reimburse External Reviewer for travel expenses
- Write to Chief/Senior Residents asking for him/her to consult with residents and prepare a confidential critique of program
- Prepare and provide documentation package to members of the Internal Review Team including:
- List of Committee Members
- Agenda for Review
- RCPSC/CFPC Pre-Survey Documentation
- Report of the last Review or Survey
- RCPSC/CFPC Specialty Training Requirements
- RCPSC/CFPC Specialty Goals and Objectives
- RCPSC/CFPC General Standards of Accreditation
- RCPSC/CFPC General Information Concerning Accreditation of Residency Programs
- Resident Report
- Distribute a report template to the Chair
- Distribute copies of Report to Program Director and Department Head
- Arrange for Chair to present the report at the Internal Review Sub-committee where the Program Director and Department Head will have the opportunity to respond