Agenda

There are set meetings which must occur; an agenda template will be provided to the program by the PGME Office. The Program will complete and return this agenda to the PGME Office.  Each program must also make the logistical arrangements for its own review, which includes identifying an external consultant within the discipline who is available on the review date.

For an in-person review, this includes a room booking, morning refreshments, and lunch.  Costs are the responsibility of the program.

CanAMS

CanAMS is the new digital accreditation system used to support all accreditation activities for the postgraduate office, residency programs and AFC programs.  A program’s submission in CanAMS will be a living document, which will be reviewed and updated prior to any internal review or accreditation activity.  In this way, the system will significantly decrease the amount of work needed prior to each accreditation review.  The program will be responsible for maintaining up to date program information on an ongoing basis.  The postgraduate office may set some internal guidelines relating to the population of CanAMS, and will provide guidance and internal deadlines for populating CanAMS instruments to ensure that they are finalized in time to meet the deadline provided by the College(s) in the context of a specific review.

In preparation for the Internal Review, each program must complete the documentation in CanAMS. The internal review team will have access to all relevant available documentation and information within the residency program’s profile instrument which will enable them to evaluate whether the standards of accreditation are being met.   All documentation in CanAMS must be updated at least 6 weeks prior to a scheduled internal/external review.  Please contact Karen Spilchen, Accreditation Program Coordinator or Jeremy Solomatenko, Educational Consultant, for further information.

Resident Report

Prior to the Internal Review, the Chief/Senior Resident of the program being reviewed will be contacted by the PGME Office asking him/her to consult with residents and complete a brief template in order to prepare a report on the program. This report will be sent to the Assistant Dean PGME, and will remain confidential, only being released to the Internal Review Team to help guide their questions on the day of the review.  It will remain confidential and will not be shared with the program. 

Review Team

Each Internal Review Team will typically be comprised of the following membership:

• Chair – who will usually be a Queen’s Program Director from another discipline.
• Resident Representative – will be a Queen’s resident from another discipline.
• External Reviewer – will usually be a Program Director from another centre in Canada who does not have a conflict of interest and has a good knowledge of the specialty being reviewed. 
• Observer (Optional) – typically another Program Director who is preparing to conduct a review.

Documents Provided

The Internal Review Team will be provided with the following information:

From the PGME Office prior to the internal review

• Schedule of meetings for the day
• RCPSC/CFPC Status Letter
• Specific Standards, Objectives of Training, and Specialty Training Requirements
• Confidential Resident Report
• General Information Concerning Accreditation of Residency Programs
• CanERA General Standards of Accreditation for Residency Programs
• Discipline-Specific Internal Review Report Template

From the Program the week prior to the meeting

• Residency Program Committee minutes and Competence Committee minutes for previous two years
• Examples of resident files (one to two trainee files per level and, if there was a trainee in difficulty, this file should be included to show the process followed)

Each member of the Internal Review Team is required to review the documentation in CanAMS before the review and prepare questions addressing the accreditation standards. This is expected to take several hours.

A series of interviews must take place with the Program Director, Division chair, Department Head, Program Assistant, teaching staff, residents, Competence Committee and the Residency Program Committee (or equivalents).  Visits to individual sites should take place as appropriate. There should be a careful assessment of the quality of the program based on the general and specific standards of accreditation as outlined by the respective College.

Meeting with the Program Director

The meeting with the Program Director is to ensure that the program structure meets the Accreditation Standards.  For example:

• How does the RPC function? Are meetings frequent enough?  How are decisions made?
• Is there appropriate representation on the Committee (i.e., Faculty members, Educators, Teaching Site Representatives, elected Resident Representatives)?
• How does the program review evaluations (of faculty, rotations, teaching sites)? What happens to this information and have any changes been made recently based on these reviews?
• Does the program provide opportunities for residents to evaluate the quality of their education?
• How do residents and faculty know how to target their teaching and learning on any given rotation?
• How are residents provided an opportunity for graded responsibility?
• Are exam Results consistent or has there been a significant reduction in performance over the last few years?  Are they consistently below national average?  If so, this may be a sign of a problem in the educational program.
• How did the program address weaknesses identified during the last review?
• What educational initiatives have been developed or updated since the last review?
• How does the program deal with residents in difficulty?  

Meeting with the Program Assistant

There are new Accreditation standards relating to the Program Assistant, and this meeting is to explore the PA’s formal role description, role within the program, and relationships with program leadership, residents and the PGME office.  Questions to consider include:

• Do you have a role description? Does it accurately reflect your work?  Is it up to date?
• Do you receive feedback, formally or informally?
• Do you receive opportunities for professional development?

Meeting with the Department Head & Division Chair

This meeting serves to identify the involvement of the Head in the Postgraduate program.  
Questions to consider include:

• Where does Postgraduate Education fall within the Department’s mandate? 
• How much does the Head support the Program Director? 
• Are adequate resources available to the program (administrative, financial, physical, material)? And if not, what efforts are being undertaken to obtain resources?

Meeting with the Residents

The reviewers will meet with all residents in the program.  In large programs it will be customary to meet separately with the junior residents and senior residents. 

This meeting is to allow the residents to identify the strengths and weaknesses of the program.  It is important to be positive and give the residents a feeling of openness.  It is suggested that the Review Team Chairperson start the discussion with an opening remark by indicating the purpose of the Internal Review and that residents should feel free to discuss any issue.  It is also important to emphasize that all discussions will remain confidential and anonymous should there be a need to identify issues in the report.  

Question the residents on the following points:

• Do they think the program is meeting the specialty training requirements?
• Are residents made aware of their educational objectives and are these being met?
• Are the residents receiving (enough) feedback?
• Do the senior residents feel that the program provides adequate opportunities to develop their skills necessary to function as a specialist physician?
• How do residents perceive the service to education balance?  Do they feel there is a good balance between education and service expectations? 
• Are there issues of intimidation and harassment in the program?
• Do residents feel there is open communication within the program? 
• Do residents know where to turn if they have an issue?
• What are the strengths of the program?

Meetings with Teaching Faculty, Residency Program Committee (or PGEC) and Competence Committee (or RAC)

These meetings are to obtain feedback on how the members of the program perceive the quality of the education and the structure of the program.   It is also an opportunity to question members of the Residency Program Committee.  Questions to consider include:

• Do faculty receive feedback on their teaching? Do they feel their teaching is valued?
• How does the program communicate with faculty in the Division/Department?
• Is the RPC dealing with issues identified at meetings and is there a follow up mechanism?
• Do members (apart from the PD) understand processes within the program, RPC and CC?
• How does the Competence Committee review assessment data and make decisions? How are these decisions communicated back to the residents?

It is recommended that a portion of this meeting should be with the Residency Program Committee without the Program Director, Division Chair, or Department Head present, to allow for frank discussion.  

Preparation for Exit Report

This meeting is to allow the Internal Review Team to come to a consensus on the apparent areas for improvement of the program and to prepare for the Exit meeting with the Program Director and the Associate Dean. The Internal Review Team should decide what they will report on during the Exit meeting.

Exit Meeting

The exit meeting is simply to obtain clarification on any issues and to outline to the Program Director the findings of the review.  Be sure to identify the areas for improvement of the Program.  Give an opportunity for the program to provide feedback on the comments you made. This will help the Internal Review Team to identify whether a specific issue is really a weakness/problem or if it was just a perception.

Prior to the review, the PGME office will provide the Chair of the Internal Review with a specialty-specific Internal Review Template which will include all of the relevant specialty-specific Standards of Accreditation down to the indicator level.  All members of the Internal Review team will contribute to the written report, which must include areas for improvement for the program and may include specific recommendations for continued development and improvements.  This report must be submitted to the Assistant Dean, PGME, no later than two weeks following the review.  The report will be sent to the Program Director with a request to respond within two weeks to correct any factual errors (response template provided).  The Internal Review and the Program Director’s Response will then be sent to the Internal Review Subcommittee for review.  The Internal Review Subcommittee will respond with a document (IRSC Response) which is intended to provide further context and constructive comments to the program to assist with the program’s efforts to address any identified AFIs.  It will also suggest next steps, which may include follow up documentation, meetings, or subsequent internal reviews.  This response is reviewed and approved by the Assistant Dean, PGME.

Internal review reports are deemed to be internal documents of the University.  However, reports of the internal review of all programs are to be provided to the survey team chair prior to the regular RCPSC/CFPC survey to enable the chair to assess the efficacy of the internal review process. Internal review reports are not available for review by the surveyors of individual programs at the time of a site visit and are not used by the College in making decisions regarding the accreditation status of individual programs.  A program may choose to provide an internal report to the College as part of its documentation to demonstrate continuous improvement efforts.

The following costs are the responsibility of the Program:

• Refreshment breaks and lunches for Internal Review Team
• Travel expenses for External Reviewer if he/she is from outside of Ontario/Quebec. Contact Karen Spilchen to review funding.

The following costs are the responsibility of the Postgraduate Medical Education Office:

• A modest honorarium for the External Reviewer (currently $1000)
• One night of accommodation for External Reviewer, unless the review encompasses a full day, in which case two nights of accommodation will be provided)
• Travel expenses for External Reviewer for travel within Ontario/Quebec

The Internal review survey team will meet with all key program stakeholders (i.e. Program Directors, Department Head, Residents, Teaching Faculty, RPC, CC, etc.) to evaluate the program based on relevant accreditation standards.

In assessing whether the program has met a requirement, the survey team will:

• Review all program information completed within CanAMS
• Ask questions where clarification is required
• On the IR Report template provided to survey team select “meets” or “does not meet”

Accreditation Context, Standards, and Tools

Click here to learn more about CanRAC, CanERA, and CanAMS

General Standards of Accreditation for Residency Programs

Standards of Accreditation for Residency Programs in Family Medicine

 Access the RCPSC Discipline-Specific Standards of Accreditation

Overview of the PGME Internal Review Process

Click here to download a copy of the PGME Internal Review Process

Training and Orientation Resources

Please refer to the PGME-Accreditation Review Orientation internal site here to view the following documents. Email Jeremy Solomatenko (JS396@queensu.ca) if you do not have access.