Research explores Competency Based Medical Education

Can medical records play a role in residency assessment? Does the interdependence of faculty and residents impact assessment? Do the entrustment scales currently fit with how faculty view resident performance in each specialty area? Could the School create a unique assessor role in postgraduate medical education and if we do, will residents find it credible?

These are just a few of the questions the research team at the Centre for Education Research & Innovation (CERI), are exploring in connection to Competency Based Medical Education (CBME).

This new research program at CERI has been kick-started by seed funding from the Dean’s Research Initiative, an AMOSO Innovations grant and three grants from the Royal College of Physicians & Surgeons of Canada.

With 10 research studies underway, seven of which are directly related to CBME assessment issues and three others that have implications for these issues, the CERI team has become the home base for this research.

One team, led by postdoctoral fellow Stefanie Sebok-Syer and CERI Director Lorelei Lingard, PhD, is looking how data from currently untapped sources could contribute to residents’ programmatic assessment.

Specifically, they are trying to determine if data from electronic health records could reflect resident competence. This project, a collaboration between CERI scientists and members of the Division of Emergency Medicine, has recently received a major funding boost with a grant from the Royal College of Physicians and Surgeons of Canada and will produce the first resident report card based in clinical data drawn from the electronic record. 

A related study is looking at the interdependence of faculty and resident performance in clinical settings. Interdependence presents a fundamental challenge to conventional assessment approaches, which assume the assessed resident performance is an independent one.

Lingard believes that this work could produce new measurement models capable of capturing both independence and interdependence, which would be a significant step forward in the assessment of performance in team-based healthcare settings.

Dr. Mark Goldszmidt and Saad Chahine, PhD, are working together to take the entrustment scale established by the Royal College of Physicians and Surgeons and adapt it to the medical context to see how well it reflects the assessment practices faculty undertake in different specialties.

“This work is showing that entrustment scales might have to be fundamentally adapted to support meaningful assessments in different clinical training environments,” said Lingard.

The team at CERI has also started a pilot project that is testing out new approaches to workplace-based assessment, with the goal of decreasing the burden on faculty members that CBME seems to threaten.

One project, led by Dr. Chris Watling and Educational Specialist Sarah Burm, PhD, recruited 12 faculty members, from various departments, and trained them in ‘hand-over’ standards. An assessment tool was developed and faculty were trained to evaluate residents with it, outside of their particular area of specialization. Following training, the group of faculty assessors observed and assessed resident hand-over in two clinical settings: critical care and paediatric medicine.

Through this project, the research team is trying to answer a number of questions including: Can assessors from outside a particular clinical area be trained to do this assessment? Do they feel comfortable? Do residents find the assessments to be credible?

With further research, CERI may be able to prove that a wide range of individuals could be trained to complete specific assessments at pivotal points in training, such as the ‘transition to discipline’ entrustable professional activities outlined for achievement during the first three months of every residency.  Could a specific assessor role become part of the medical education team? More pilot studies and future research will help to answer that critical question.

As the transition to CBME continues, more questions will be posed, and no doubt the CERI team, with their collaborators from across the School will be ready to take them on as they explore how best to enrich medical education.