Internal Review Process | School of Medicine

If you are a Program Director or Program Assistant preparing for an Internal Review, please click here for document access

The internal review process is an integral part of continuous quality improvement in postgraduate medical education at Queen’s. The goals are two-fold:

To prepare programs for the regular on-site accreditation visits from the Royal College of Physicians and Surgeons of Canada (RCPSC) and the College of Family Physicians of Canada (CFPC).
To drive continuous quality improvement at both the individual program level and at the institution level.

The Royal College of Physicians and Surgeons of Canada (RCPSC) and the College of Family Physicians of Canada (CFPC) conduct regular on-site accreditation visits to ensure the quality of the residency programs at Queen’s University. The accreditation process follows an 8-year cycle which includes the on-site survey, regularly scheduled internal reviews, and any update reports (APORs) or external reviews of the residency training programs mandated by the Colleges.

The Postgraduate Medical Education office (PGME) at Queen’s is responsible for conducting regular mid-cycle internal reviews, and ensuring identification and follow up of any areas for improvement for each residency program. At Queen’s, this work is undertaken by the Internal Review Subcommittee (IRSC), a subcommittee of the Postgraduate Medical Education Committee (PGMEC). The goal of the IRSC is to provide individual programs with a high-quality internal review and a constructive response to their internal review, to help drive continuous quality improvement at the individual program level. They also identify leading practices or innovations within programs, and common areas for improvement across programs, to bring to PGMEC so that best practices and solutions can be shared.

Looking towards the next Accreditation site visit to Queen’s in 2026, the internal review process is focused on ensuring programs meet the new CanERA Standards of Accreditation, as this will be the first on site visit to Queen’s applying the new Standards.

Objectives

The objectives of the Internal Review are as follows:

To provide the program an opportunity to reflect on the current state of their program in relation to the new Standards of Accreditation
To objectively assess the strengths and areas for improvement (AFIs) for each program
To provide context and constructive comments (in the IRSC Response to Internal Review) to help programs consider how to best address AFIs
To drive ongoing quality improvement in each residency program

Preparing for the Internal Review

Agenda

There are set meetings which must occur; an agenda template will be provided to the program by the PGME Office. The Program will complete and return this agenda to the PGME Office. Each program must also make the logistical arrangements for its own review, which includes identifying an external consultant within the discipline who is available on the review date.

For an in-person review, this includes a room booking, morning refreshments, and lunch. Costs are the responsibility of the program.

CanAMS

CanAMS is the new digital accreditation system used to support all accreditation activities for the postgraduate office, residency programs and AFC programs. A program’s submission in CanAMS will be a living document, which will be reviewed and updated prior to any internal review or accreditation activity. In this way, the system will significantly decrease the amount of work needed prior to each accreditation review. The program will be responsible for maintaining up to date program information on an ongoing basis. The postgraduate office may set some internal guidelines relating to the population of CanAMS, and will provide guidance and internal deadlines for populating CanAMS instruments to ensure that they are finalized in time to meet the deadline provided by the College(s) in the context of a specific review.

In preparation for the Internal Review, each program must complete the documentation in CanAMS. The internal review team will have access to all relevant available documentation and information within the residency program’s profile instrument which will enable them to evaluate whether the standards of accreditation are being met. All documentation in CanAMS must be updated at least 6 weeks prior to a scheduled internal/external review. Please contact Karen Spilchen, Accreditation Program Coordinator or Jeremy Solomatenko, Educational Consultant, for further information.

Resident Report

Prior to the Internal Review, the Chief/Senior Resident of the program being reviewed will be contacted by the PGME Office asking him/her to consult with residents and complete a brief template in order to prepare a report on the program. This report will be sent to the Associate Dean PGME, and will remain confidential, only being released to the Internal Review Team to help guide their questions on the day of the review. It will remain confidential and will not be shared with the program.

Review Team

Each Internal Review Team will typically be comprised of the following membership:

Chair – who will usually be a Queen’s Program Director from another discipline.
Resident Representative – will be a Queen’s resident from another discipline.
External Reviewer – will usually be a Program Director from another centre in Canada who does not have a conflict of interest and has a good knowledge of the specialty being reviewed.
Observer (Optional) – typically another Program Director who is preparing to conduct a review.

Documents Provided

The Internal Review Team will be provided with the following information:

From the PGME Office prior to the internal review

Schedule of meetings for the day
RCPSC/CFPC Status Letter
Specific Standards, Objectives of Training, and Specialty Training Requirements
Confidential Resident Report
General Information Concerning Accreditation of Residency Programs
CanERA General Standards of Accreditation for Residency Programs
Discipline-Specific Internal Review Report Template

From the Program the week prior to the meeting

Residency Program Committee minutes and Competence Committee minutes for previous two years
Examples of resident files (one to two trainee files per level and, if there was a trainee in difficulty, this file should be included to show the process followed)

Each member of the Internal Review Team is required to review the documentation in CanAMS before the review and prepare questions addressing the accreditation standards. This is expected to take several hours.

Meetings

A series of interviews must take place with the Program Director, Division chair, Department Head, Program Assistant, teaching staff, residents, Competence Committee and the Residency Program Committee (or equivalents). Visits to individual sites should take place as appropriate. There should be a careful assessment of the quality of the program based on the general and specific standards of accreditation as outlined by the respective College.

Meeting with the Program Director

The meeting with the Program Director is to ensure that the program structure meets the Accreditation Standards. For example:

How does the RPC function? Are meetings frequent enough? How are decisions made?
Is there appropriate representation on the Committee (i.e., Faculty members, Educators, Teaching Site Representatives, elected Resident Representatives)?
How does the program review evaluations (of faculty, rotations, teaching sites)? What happens to this information and have any changes been made recently based on these reviews?
Does the program provide opportunities for residents to evaluate the quality of their education?
How do residents and faculty know how to target their teaching and learning on any given rotation?
How are residents provided an opportunity for graded responsibility?
Are exam Results consistent or has there been a significant reduction in performance over the last few years? Are they consistently below national average? If so, this may be a sign of a problem in the educational program.
How did the program address weaknesses identified during the last review?
What educational initiatives have been developed or updated since the last review?
How does the program deal with residents in difficulty?

Meeting with the Program Assistant

There are new Accreditation standards relating to the Program Assistant, and this meeting is to explore the PA’s formal role description, role within the program, and relationships with program leadership, residents and the PGME office. Questions to consider include:

Do you have a role description? Does it accurately reflect your work? Is it up to date?
Do you receive feedback, formally or informally?
Do you receive opportunities for professional development?

Meeting with the Department Head & Division Chair

This meeting serves to identify the involvement of the Head in the Postgraduate program.
Questions to consider include:

Where does Postgraduate Education fall within the Department’s mandate?
How much does the Head support the Program Director?
Are adequate resources available to the program (administrative, financial, physical, material)? And if not, what efforts are being undertaken to obtain resources?

Meeting with the Residents

The reviewers will meet with all residents in the program. In large programs it will be customary to meet separately with the junior residents and senior residents.

This meeting is to allow the residents to identify the strengths and weaknesses of the program. It is important to be positive and give the residents a feeling of openness. It is suggested that the Review Team Chairperson start the discussion with an opening remark by indicating the purpose of the Internal Review and that residents should feel free to discuss any issue. It is also important to emphasize that all discussions will remain confidential and anonymous should there be a need to identify issues in the report.

Question the residents on the following points:

Do they think the program is meeting the specialty training requirements?
Are residents made aware of their educational objectives and are these being met?
Are the residents receiving (enough) feedback?
Do the senior residents feel that the program provides adequate opportunities to develop their skills necessary to function as a specialist physician?
How do residents perceive the service to education balance? Do they feel there is a good balance between education and service expectations?
Are there issues of intimidation and harassment in the program?
Do residents feel there is open communication within the program?
Do residents know where to turn if they have an issue?
What are the strengths of the program?

Meetings with Teaching Faculty, Residency Program Committee (or PGEC) and Competence Committee (or RAC)

These meetings are to obtain feedback on how the members of the program perceive the quality of the education and the structure of the program. It is also an opportunity to question members of the Residency Program Committee. Questions to consider include:

Do faculty receive feedback on their teaching? Do they feel their teaching is valued?
How does the program communicate with faculty in the Division/Department?
Is the RPC dealing with issues identified at meetings and is there a follow up mechanism?
Do members (apart from the PD) understand processes within the program, RPC and CC?
How does the Competence Committee review assessment data and make decisions? How are these decisions communicated back to the residents?

It is recommended that a portion of this meeting should be with the Residency Program Committee without the Program Director, Division Chair, or Department Head present, to allow for frank discussion.

Preparation for Exit Report

This meeting is to allow the Internal Review Team to come to a consensus on the apparent areas for improvement of the program and to prepare for the Exit meeting with the Program Director and the Associate Dean. The Internal Review Team should decide what they will report on during the Exit meeting.

Exit Meeting

The exit meeting is simply to obtain clarification on any issues and to outline to the Program Director the findings of the review. Be sure to identify the areas for improvement of the Program. Give an opportunity for the program to provide feedback on the comments you made. This will help the Internal Review Team to identify whether a specific issue is really a weakness/problem or if it was just a perception.

The Report

Prior to the review, the PGME office will provide the Chair of the Internal Review with a specialty-specific Internal Review Template which will include all of the relevant specialty-specific Standards of Accreditation down to the indicator level. All members of the Internal Review team will contribute to the written report, which must include areas for improvement for the program and may include specific recommendations for continued development and improvements. This report must be submitted to the Associate Dean, PGME, no later than two weeks following the review. The report will be sent to the Program Director with a request to respond within two weeks to correct any factual errors (response template provided). The Internal Review and the Program Director’s Response will then be sent to the Internal Review Subcommittee for review. The Internal Review Subcommittee will respond with a document (IRSC Response) which is intended to provide further context and constructive comments to the program to assist with the program’s efforts to address any identified AFIs. It will also suggest next steps, which may include follow up documentation, meetings, or subsequent internal reviews. This response is reviewed and approved by the Associate Dean, PGME.

Internal review reports are deemed to be internal documents of the University. However, reports of the internal review of all programs are to be provided to the survey team chair prior to the regular RCPSC/CFPC survey to enable the chair to assess the efficacy of the internal review process. Internal review reports are not available for review by the surveyors of individual programs at the time of a site visit and are not used by the College in making decisions regarding the accreditation status of individual programs. A program may choose to provide an internal report to the College as part of its documentation to demonstrate continuous improvement efforts.

Costs

The following costs are the responsibility of the Program:

Refreshment breaks and lunches for Internal Review Team
Travel expenses for External Reviewer if he/she is from outside of Ontario/Quebec. Contact Karen Spilchen to review funding.

The following costs are the responsibility of the Postgraduate Medical Education Office:

A modest honorarium for the External Reviewer (currently $1000)
One night of accommodation for External Reviewer, unless the review encompasses a full day, in which case two nights of accommodation will be provided)
Travel expenses for External Reviewer for travel within Ontario/Quebec

Timelines

The Internal review survey team will meet with all key program stakeholders (i.e. Program Directors, Department Head, Residents, Teaching Faculty, RPC, CC, etc.) to evaluate the program based on relevant accreditation standards.

In assessing whether the program has met a requirement, the survey team will:

Review all program information completed within CanAMS
Ask questions where clarification is required
On the IR Report template provided to survey team select “meets” or “does not meet”

Six months prior to the Internal Review	Schedule template distributed to Program Assistants [PGME] Consult with potential External Reviewers [Program Director] Set date, ensuring availability of External Reviewer Department Head, Division Chair, Faculty, and Residents [Program Director]
Four Months prior to the Internal Review	External Reviewer contact information provided to PGME [Program Director/Program Assistant] Preparations begun for internal review [Program] Department Head, Division Chair, Faculty and Residents provided with details of visit [Program Director/Program Assistant] Template for Resident Report sent to Chief Resident(s) for completion [PGME]
Two months to four weeks prior to Internal Review	Four weeks before IR date, programs should share their OneDrive folders with Karen Spilchen (karen.spilchen@queensu.ca) and Jeremy Solomatenko (js396@queensu.ca) Instruments in CanAMS to be completed for review by PGME office at least 6 weeks prior to IR. Notify Karen Spilchen (karen.spilchen@queensu.ca) as soon as this is complete [Program Director/Program Assistant] Internal Review team is provided access to CanAMS and the program's OneDrive folder [Program] Resident Report due PGME Office [Chief/Senior Resident]
Two to four weeks before	Internal Review team is provided access to CanAMS along with additional documents (Decision letter, Schedule of meetings for the day, Resident Report, Sample answers, Report Template, etc.) Virtual meetings set up as required
One week before	For a virtual review, establish process for virtual review of minutes and resident assessment documents and provide links to internal review team as required [PGME to provide guidance/Program]
Internal Review Day	Program Assistant is available should there be any questions from review team and responsible for all meetings throughout the day
Two weeks following review	Report Template to be completed and returned to PG Office [Chair of the Internal Review] The Report is shared with the program and the PD is given the opportunity to respond in brief to the report identifying any errors using the Program Response to IR template.
1-4 Months following review	The IRSC will review the Report and findings /PD Response to IR and prepares an IRSC IR Response for the program with constructive comments and context around areas for improvement with recommendations for further follow-up. PGME Associate Dean will review, and add to or sign off on recommendations.
Follow up as per IRSC / PGME recommendations	The PG Office and IRSC follow-up with the program to monitor areas for improvement based on the internal review feedback. Additional guidance and support is available to the program as required.

Training and Orientation

Accreditation Context, Standards, and Tools

Click here to learn more about CanRAC, CanERA, and CanAMS

General Standards of Accreditation for Residency Programs

Standards of Accreditation for Residency Programs in Family Medicine

Access the RCPSC Discipline-Specific Standards of Accreditation

Overview of the PGME Internal Review Process

Click here to download a copy of the PGME Internal Review Process

Training and Orientation Resources

Please refer to the PGME-Accreditation Review Orientation internal site here to view the following documents. Email Jeremy Solomatenko (JS396@queensu.ca) if you do not have access.

Program Directors	Program Assistants	PGME	Reviewers
CanAMS Training	CanAMS Training	CanAMS Training	CanAMS Training
PD Checklist	PA Checklist	CanAMS Tips	Reviewer's Checklist
CanAMS Tips	PA Interview Prep		Reviewer's Tips
	Sample IR Schedule		OneDrive Review Team Guide
	OneDrive Organization
	CanAMS Tips