Unlearning in Clinical Trials – the cure to repeating our mistakes

How many stories have you heard where mistakes – made by clinical research sites – caused problems in a study, or even worse, patient injury? The practice of medicine is inherently risky given the consequences of mistakes or misinformation. This is even more the case in research, where we are layering on an extra level of complexity with the protocol. Often, we ask investigators to do things that are counterintuitive to their medical training, such as curbing bedside manner and not providing emotional support to a patient in a study if the resulting placebo effect could impact the end points.

Despite the difficulties and challenges investigators face, sponsors still provide only the most basic support in terms of helping them understand the protocol and conduct a successful study. Study teams rely heavily on investigator meetings and slide decks to train site staff and not even all the site staff, often just the investigators and perhaps one study coordinator. These people are not professional trainers and they are very busy, so why do we expect them to successfully train the rest of their staff? I hate to break it to everyone, but face-to-face lectures are a 3,000-year-old training technique that has been proven time and again to be ineffective (Bajak, 2014). As the Chinese proverb goes:

What I hear I forget,

what I see I remember,

what I do I understand.

Check out the 2014 study titled, “Lectures aren’t just boring, they’re Ineffective, too”. A recognized improvement to standard lecture-style training is simulation (Cook, Hatala, Brydges, 2011), where there is “strong evidence demonstrating improvement in learners’ knowledge, skills, and behaviors.” Even across cultures, simulated training has been shown to be rated as a valuable learning experience by learners and is linked to better academic performance (Williams, Abel, Khasawneh, Ross, Levett-Jones, 2016). This means the message you are teaching doesn’t break down when you open those sites in Croatia or the Ukraine.

I understand that with hundreds of millions of dollars at stake, it can be scary to use new technology to improve the clinical trials process, but frankly, modern training and human performance improvement techniques such as simulation are NOT new. If I were running a global phase 3 study, I would find it much scarier to use an approach that is proven not to work rather than using technology that is held as the gold standard by multi-billion dollar industries such as airlines, the military, and NASA.

It comes down to unlearning. We need to unlearn the old, ineffective, ways of doing things in order to make space for new methods and approaches. Just because something is being done the way it has always been done does not mean that it is lower risk. If you jump off your roof 10 times, you may only sprain an ankle once, but is it really more risky to try a ladder? It’s like John Cage said, “I can’t understand why people are frightened of new ideas. I’m frightened of the old ones.”

References

Bajak, A. (2014). Lectures aren’t just boring, they’re Ineffective, too, study finds. Retrieved from https://www.sciencemag.org/news/2014/05/lectures-arent-just-boring-theyre-ineffective-too-study-finds

Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. 2011; 306:978-988.

Williams B, Abel C, Khasawneh E, Ross L, Levett-Jones T. Simulation experiences of paramedic students: a cross-cultural examination. Adv Med Educ Pract. 2016; 7():181-6.

Chatman, J. The study is published in The Leadership Quarterly, Vol. 21, Issue 1 (2010), 104-113.