Measuring Training ROI in Clinical Research

Clinical trials are vital to the future of medicine and pharmaceuticals. Many of them are literally matters of life and death, studying the effect of potentially life-saving drugs or researching the impact of devastating diseases. So, why is it that the training of study moderators and site teams is treated so nonchalantly?

The traditional methods used to train these teams are extremely outdated, inefficient, and ineffective. This leaves site teams unprepared for the realities of the trial, increases protocol deviations, and drastically increases the overall cost of the study.

But many sponsors don’t even measure the ROI (return on investment) of the training they administer to determine how it’s impacting their trials and bottom line. Unfortunately, CRO-managed studies actually benefit from increased deviations because they are also tasked with fixing them, which lengthens the study and makes them more money. Recent data from the Tufts Center for the Study of Drug Development revealed that a typical phase three trial has 120 protocol deviations affecting 30% of the study subjects, but CRO-managed trials have 64% higher protocol deviations affecting 70.8% of the study subjects. This is not a coincidence. Each amendment made to a trial costs approximately $500,000 to correct, so there is little incentive for CROs to optimize training and reduce errors despite the high stakes of many clinical trials.

Nevertheless, as sponsors evaluate all elements of their clinical research, there is a reliable method for determining the ROI of their training.

The Kirkpatrick Model

---

The Kirkpatrick Model was developed in the 1950s by Donald Kirkpatrick, a professor and training specialist, to accurately measure training effectiveness. There are four levels sponsors can use to evaluate their training methodology:

• Level 1 – Reaction
  The degree to which participants find the training favorable, engaging, and relevant to their jobs. The current training in our industry is severely lacking, consisting of mind-numbing powerpoints and lectures that don’t do anything to engage the participant.
• Level 2 – Learning
  The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation. Current training methods are very ineffective at building retention, leading to higher rates of protocol deviations during the trial.
• Level 3 – Behavior
  The degree to which participants apply what they learned during training when they are back on the job. If they are prepared with adequate training, site teams will be able to retain information better and act more appropriately in situations, reducing errors across the board.
• Level 4 – Results
  The degree to which targeted outcomes occur as a result of the training and the support and accountability package. This is the level where ROI can be measured.

Unfortunately, sponsors don’t often measure the ROI of training at all. In fact, they barely test site teams for retention after training is administered. The only standard sponsors are required to meet by the FDA is simply to administer the training, not determine how well it works.

But deviations can have tremendous impacts on the bottom line of a trial and on the well-being of the patients involved, so even without calculating the exact ROI, it’s obvious that improved training would be of benefit to sponsors in the end, even if it calls for higher initial investment.

The ROI of Simulation-Based Training

---

Take the Boeing 737 Max, for example. Normally, pilots are required to complete thousands of hours of simulation training to fly, but they were not required to do so for this new aircraft due to the high cost. But when faced with emergency crash situations, the pilots were unable to perform basic procedures that would have shown up in simulation training and could have saved hundreds of lives.

Sure, it would have cost Boeing several hundred million dollars to train these pilots in a simulator, but what is that compared to the cost of those devastating crashes?

Why then are clinical trials, often testing life-saving procedures and putting patient health at risk, treated differently? Well, many sponsors don’t even know it’s possible to measure the ROI of training, and simulation-based training is a larger upfront cost without an immediate return, which deters many from utilizing it.

Nevertheless, the industry is slowly changing to adopt simulation-based training more broadly as the ROI is discovered. Good training should result in about a 5,000% ROI, and in fact, we estimate that Pro-ficiency’s simulation training solutions will save $1 – 10 million per study by better preparing site teams, reducing deviations, and mitigating the need for costly amendments to be made mid-study.

All training should be designed to have a positive business impact. The more that sponsors adopt simulation-based training, the more data they will be able to gather and analyze, allowing them to realize the tremendous ROI associated with training their site teams using real-world scenarios.

To learn more visit: https://www.pro-ficiency.com/simulation-training/