COVID-19 challenged the medical community and highlighted the need to review how trainees are assessed. With an opportunity to rethink the validity, efficacy and intention of high-stakes assessments, an ICRE panel of experts debated how to change the system.

The latest International Conference of Residency Education (ICRE) brought together close to 1,200 clinical educators, surgeons, physicians, residents and medical students worldwide to share ideas, challenges, innovations, and provide advanced training.

In the session: High-stakes Assessments: Redefining the Role in a Post-Pandemic World, a power panel of international experts tackled difficult conversations around the functions and validity of the end-of-residency exams and final assessments in today's environment.

Dr. Flavia Senkubuge, President, Colleges of Medicine South Africa (CMSA), Dr. Aimee Charnell, Surgical Research Fellow at Leeds Institute of Medical Education; Dr. Adam Sawatsky, Associate Professor of Medicine, Division of General Internal Medicine, Mayo Clinic and Dr. Farhan Bhanji, Professor of Pediatrics at McGill University joined moderator Dr. Viren Naik, Director of Anesthesiology at the University of Ottawa and Director of Assessment at the Royal College of Physicians and Surgeons of Canada for a provocative discussion that explored examination benefits, issues and options for the future.

Certifications, yes or no?

Dr. Naik began the discussion by asking this controversial question. As the panel shared their varying thoughts on the topic, a few key points in favour of high-stakes assessments rose to the surface:

• Examinations serve as accountability - Dr. Bhanji raised the point that as a self-regulating profession, medical professionals must have accountability. Exams may help serve that purpose in many different contexts and environments. “There is evidence to back this up," he explained. “In some contexts, such as in the United States, where people are able to practice without being certified, it's clear that those who are uncertified are more likely to be called up in front of their state boards, have their licenses revoked and have poor patient outcomes."
• The public expects a level of rigour - Dr. Senkubuge asserted that the public still expects a doctor or specialist to reach a milestone achievement in order to be certified. “We should never lose focus on the fact that we do need to manage whatever conversation we are having with that public expectation," she said, adding “I am not a proponent of a big slap-bang one-time event, but I am also cognizant of the fact that if we put on our public hat, the public expects you have passed or have achieved this particular event in order to be certified."
• Assessments may provide a global benchmark - Dr. Naik suggested examinations may serve as a calibration in a global world, asking if they have a role on the international stage. Dr. Senkubuge believes there is, stating that if someone new is coming in from another country, an international exam helps gauge whether they are a safe practitioner or specialist. She added, however, that the examination is only one side of the story and a doctor moving from Europe to South Africa, for example, is entering a vastly different health system, and understanding and acclimatizing to the new system is equally important.
• Assessments may predict how people perform in clinical practice - Dr. Bhanji also explained that studies have shown correlations between the performance of international medical graduates on exams and their patient outcomes. The opposite is true as well: “How people perform in clinical practice training predicts how well they do on an exam," he said.

Flip side of the coin: An examination of the issues

On the other hand, the panel suggested there are issues with the current examination practices that have an opportunity to be reassessed as the pivots made while facing COVID restrictions have altered the exam landscape. Here are some of the primary questions about high-stakes assessments that emerged from the discussion.

• What are the consequences for trainees? Researcher Dr. Adam Sawatsky discussed the implications and consequences of assessments on trainees. “The body of evidence was there — and building before COVID ever came — that the consequences of these evaluations are tough for learners," he said, adding that COVID has given them an opportunity to rethink examinations, recognizing they're not doing everything they were intended to.
• Are examinations fair and unbiased? How are questions chosen? How is each question scored? How do you set standards? There is an inherent level of subjectivity to assessments, Dr. Charnell said, that is difficult to address. “Many have argued that exams are sexist, racist, or discriminate against people with caring responsibilities. There are systemic problems with these examinations." As a result, many trainees no longer trust exams are passing and failing the right people.
• Should assessments be one-day terminal events? The panellists agreed that one-day examinations are problematic, as assessments at the end of a program don't necessarily identify the best trainees but rather only weed out those who aren't performing to the test's standards. To that point, as Dr. Bhanji pointed out how instructors often know early on who is going to perform well on the exam and who won't. “100 times an instructor knows who is going to fail the exam," he said. “We have hard data showing you can predict who's going to fail by the second year — often, it's not surprising."

Assessing the way forward for high-stakes assessments

Determining the way forward is difficult, and getting consensus is even harder, said Dr. Bhanji. Here are some potential solutions offered by the panelists to enhance equity, humanity and quality in student and trainee assessments.

1. Competency-based education

Dr. Sawatsky believes in competency-based education and a move towards coaching to help trainees get a holistic, professional identity. He explained the approach as “the kind that embraces the standardization of competencies not only as we define them, but also the individualization of those competencies."

2. Embrace subjectivity

Can any assessment be 100 per cent objective in medicine? Dr. Sawatsky doesn't believe it can. Instead, he feels what is needed is to acknowledge some of the inherent subjectivity. “I think part of the problem with trying to be objective is measuring things we believe we can objectively measure — such as medical knowledge. That's only a small piece of everything it means to be a physician." He feels instead it's important to balance what can be objectively assessed with what cannot.

3. Adopt recertification

As the field of medicine evolves, physicians must continue to learn. “One of the errors we make is that we focus all our attention on medical school and residency and forget that there are 35 years of continued professional development," said Dr. Charnell.

Creating a culture where educational assessments are objective, fair, equitable, humane and doing what they're intended to do is the ultimate goal of clinical educators. COVID has highlighted the need to reflect and evaluate paths forward for the good of healthcare professionals and the future of the industry. Recognizing the humanity of residents and teachers will enable a healthier future for all medical professionals. “We need to dismantle the survival mentality," said Dr. Senkubuge. “The training period should be a time of reflection, of learning, but most of all growth. Not survival."

* Diane Amato is a Toronto-based freelance writer who loves to talk about finances, healthcare, travel and technology.

This article originally appeared on the RBC Healthcare - Advice & Learning