Vaccines prevent 2-3 million deaths per year. Unlike most medicines, vaccines don’t treat disease, but they can prevent it from spreading and making people sick by producing complete or partial immunity to a specific disease. And unlike medicines, a vaccine’s success is not based on efficacy and safety alone—public trust is paramount. Scientists estimate 60% of the population would need to have significant immunity in order to achieve “herd immunity” (where enough of the community is immune to prevent person-to-person spread).
In response to COVID-19, many in the global healthcare and life sciences industry are focused on developing vaccines as a powerful tool for stemming the pandemic. While vaccine development usually takes years, scientists and public health authorities are moving faster than ever before—at “warp speed”—with the goal of developing a safe, effective vaccine in less than a year.
Clinical trials for multiple vaccine candidates are enrolling participants at many sites across the world, with major participation in the U.S. The U.S. is not only the source of major biomedical innovation, but unfortunately we’re one of the few countries producing large numbers of infections, a critical factor in researching the efficacy of a vaccine candidate.
Growing mistrust of vaccines represents a serious challenge. Especially in light of the speed of the research and development timeline, scientific transparency and increased public awareness of the clinical research process is critical. The Emergency Use Authorization authority of the FDA enhances concerns, because vaccines can be approved based on potential benefits and risks as well as proven benefits and risks. In usual times, the benefits and risks must be proven rather than “potential.” While the enormous mortality, illness and economic losses from COVID-19 justify this emergency authority for the FDA, public confidence becomes even more important.
As a former FDA Commissioner and long-time clinical trialist and clinician, I have been asked about why I have confidence that the system will produce a reliable result from the ongoing clinical trials. These discussions have reinforced the importance of providing more information about clinical trials to both biomedical professionals and the general public.
The recent publication of the first three study protocols for the vaccine trials provides useful details to bolster confidence that the trials are being conducted rigorously. So how are the trials set up, and what entities and safeguards are in place?
How the clinical trials work
The FDA has determined that a viable COVID-19 vaccine must reduce new infections by at least 50% compared to a placebo in the trials. The trials are set up in a “classical fashion,” with all the standard division of responsibility and safeguards needed to allow researchers to draw valid scientific conclusions. The research occurs in phases, starting with pre-clinical research, then studies involving animal testing (Phase 0), followed by human trials focused on preliminary evidence of safety and efficacy (Phase I), and then by larger trials for safety and efficacy (or how effective the vaccine is at stimulating the immune system) (Phases II and III). A vaccine that fails at any of these stages cannot move forward. All of the developmental phases culminate with a large, pivotal Phase III Trial that is intended to prove safety and efficacy.
Another critical element to achieving scientific validity is organizing the trials to ensure that bias—intellectual, financial or political—is prevented from leading to untrue results, putting both trial participants and future patients at risk. One way we reduce bias is by ensuring that trial participant assignment to active vaccine or placebo is randomly allocated, and that researchers and participants are “blinded.”
This means participants and researchers are unaware of which individuals have received the true vaccine or the control placebo until the study is finished. There are mechanisms to unblind individuals whether they receive a true vaccine or placebo to protect them, especially if they experience severe side effects that could be caused by the experimental vaccine. Research sites, led by experienced and qualified physicians, are responsible for getting informed consent, administering the vaccine or placebo, collecting data while monitoring participants’ well-being and attending to the needs of participants should health issues arise during the trial.
Researchers and participants knowing who has the vaccine or the placebo is only one of many possible sources of bias. The Food and Drug Administration (FDA), National Institutes of Health (NIH) and other trial experts have learned how to guard against these biases by designing the trials correctly.
In addition to trial design, there are numerous entities who play a role in ensuring the validity of trials. Here is a list of additional entities in place for planning, conducting, analyzing and interpreting COVID-19 vaccine trials:
Each manufacturer, which could be a pharma or biotech company, that has moved a vaccine candidate through the first few phases is known as the “sponsor” of the trial. Each sponsor is paired with a clinical research organization (CRO) or uses an internal equivalent of a CRO for trial operations.
- What they do: CROs directly coordinate the work of the research sites in each trial and manage the trial database. The CROs are legally considered to be agents of the sponsor.
- Access to data: The trial sponsors handle regulatory affairs and they are responsible for distributing and marketing should the vaccine prove safe and effective, but they have no knowledge of the trial results until they are unblinded for primary analysis. This helps assure that there is no manipulation of the trial results for financial or reputational gain.
Each trial has a Data Monitoring Committee (DMC), which is a standard practice for major trials with clinically important outcomes.
- What they do: The DMC consists of experts in infectious disease, vaccine trials, biostatistics and ethics. The group receives reports from the Data Coordinating Center at the CRO that enables it to track the progress of the trial and to evaluate unblinded data at regular intervals.
- Access to data: This group is responsible for monitoring the trial and data so no bias can creep in.The DMC analyzes trial data at pre-specified intervals, ranging from after ⅓ and ⅔ of total infections have occurred to ¼, half, ¾ through the number of infections, to assess efficacy of the vaccine. The DMC could recommend stopping the trial at one of these intervals for either a large treatment effect or a finding of futility (no chance of showing the needed treatment effect). The primary factor in the DMC decision is based on a prespecified statistical plan. At these stages, a much larger effect must be observed to declare efficacy and stop early than is required at the end of the trial (commonly known as a “statistical penalty”). These decisions are not just formulaic—the committee must consider all information at hand. The DMC is sworn to confidentiality until the trial is over.
Institutional Review Boards oversee the ethical conduct of each trial. There are both central IRBs paired with each trial, to monitor the overall trial, and local IRBs at every research site to oversee the local conduct of the research.
- What they do: The IRB oversees the conduct of researchers and assures that processes are in place to protect participants.
- Access to data: The IRBs are made up of experts in clinical research and ethics with no vested interest in the trial results—their responsibility is to the research participants. They do not have access to the data during the conduct of the trial, except for reporting of adverse events.
The FDA Vaccines and Related Biological Products Advisory Committee (VRBAC) reviews applications from the manufacturer prior to consideration of approval. VRBAC, like all FDA Advisory Committees, is only advisory; it does not make the decisions. In preparation for Advisory Committee meetings the Center for Biologics Evaluation and Research (CBER) staff analyzes the data independently of the manufacturer to provide perspective for public consumption so that the committee can discuss issues that might affect the final decision.
- What they do: This committee includes experts in the field, representing medical, scientific, statistical and patient perspectives. They have a public deliberation about candidate vaccines and the trials that evaluate their safety and efficacy. The FDA full-time staff is responsible for making the final decision about whether the vaccine can be marketed and distributed, not the advisory committee. FDA does not commonly make a decision that is different than Advisory Committee recommendations, but it does happen.
- Access to data: The FDA will have access to all data including the raw data, at the end of the trial. VRBAC and the public will have access to reports from the data prepared by the FDA and the sponsors.
The Centers for Disease Control and Prevention Advisory Committee on Immunization Practices (ACIP) includes public health and medical experts.
- What they do: ACIP advises the CDC about the use of vaccines in the U.S. civilian population.
- Access to data: ACIP will have full access to the data and conclusions from the FDA and sponsor(s) at the end of the trial.
Operation Warp Speed (OWS) is a special organization across the Federal government, formed specifically for COVID-19 because we are in an official national emergency.
- What they do: OWS officials are aware of all the operations of the trials and have accountability for the overall strategy and funding allocation, but they do not know the trial results as the studies proceed.
- Access to data: OWS leaders are “hands off” on the data and treatment assignment to avoid a biased, overly optimistic interpretation.
The Accelerating Covid-19 Therapeutic Interventions and Vaccines (ACTIV) trial network is being operated by the National Institutes of Health (NIH) with a highly experienced and savvy group of clinical trialists who have extensive experience with NIH-funded clinical trials.
- What they do: They are managing the COVID-19 trial operations at a high level and organize the research sites for the trials in conjunction with the sponsors. As opposed to Warp Speed, which is in place to integrate logistics across the public and private sectors, the NIH is the main organization accountable for the science of the trials.
- Access to trial data: By design, NIH officials have no knowledge of the evolving trial results. They need to be unbiased as they officiate the design and conduct of the trials and monitor the evolving science as the trials proceed.
It is important to reiterate that the full-time CBER staff has the primary responsibility for making the decision about whether a vaccine can be marketed. The rest of this trial machinery is intended to create an unbiased operation that will yield reliable findings at the end of the trial. Almost never in the history of the FDA has a political appointee over-ruled the Center Director in a decision about an individual product. Setting a precedent for political meddling with specific FDA product decisions would jeopardize the integrity of future FDA decisions, and both Peter Marks, the Center Director and Stephen Hahn, the Commissioner have publicly committed to “staying with the science.”
From the initial design to the final deployment, the vaccine clinical trials system is configured to have layers of accountability to prevent contamination of the study and should be impervious to biased influence by political, corporate or medical ideological meddling. With so many checks and balances in place, built up through decades of experience, any effort to alter the science will become a topic for public scrutiny.
Recently, concern has been expressed about the possibility that the Secretary of Health and Human Services or the President could overrule or preempt the Center Director and the Commissioner. While this is possible as the FDA is part of the Executive Branch of government, and it is a “chain of command”, ultimately a vaccine will need the confidence of the public, including the doctors, nurses and pharmacists who administer the vaccine. I am confident scientific facts about the vaccine will be known, allowing biomedical professionals to form a consensus about the balance of benefits and risks that will drive public perception. Efforts to override the consensus of scientists and health professionals will likely not succeed because the trials will be scientifically sound, the data will be publicly available and most people will not take a vaccine without a positive assertion from their trusted health professional. Hopefully, the trials will be dramatically positive and we can all have confidence not only in the trials, but also in the decisions based on the trial results.
- Robert M. Califf, MD, MACC, Head of Clinical Policy and Strategy for Verily and Google Health