How do you know a vaccine works?

Nearly a year after the outbreak of the COVID-19 epidemic, the first vaccines against it were recently approved for use. Currently two companies – Pfizer and Moderna – have successfully completed all phases of clinical trials in their vaccines against the pathogen, the coronary virus SARS-CoV-2, and more companies expect to complete further vaccine development soon. But how do you even know how effective a vaccine is?

A vaccine is designed to bring our immune system together with the cause of the disease in a controlled way, that is, when it is weakened, dead or only a small part of it is introduced into the body. This encounter leaves in our body a long-term immune memory. If we are then exposed to the real pathogen, we will already have antibodies and immune cells that will know to detect the invader and neutralize it quickly, before it causes us harm.

To deal with the corona plague many vaccines have been developed around the world. As of this writing, about 80 of them are in clinical research stages, and more than 150 in earlier stages. Before a new vaccine is launched, animal experiments must be performed, in which the safety and effectiveness of the vaccine are examined, as well as the side effects caused by different concentrations of the substance. If he successfully passed these steps, clinical trials in humans continue.

In the first phase of clinical trials the vaccine is given to healthy volunteers to make sure the vaccine is safe and does not cause serious side effects. In more advanced stages, the effectiveness of the vaccine and its contribution to preventing infection and eradicating the symptoms of the disease are examined. In these stages, two groups are compared: the treatment group, which receives the vaccine, and the control group, which receives a placebo vaccine (placebo) without an active substance. The selection of the subjects into groups is done at random and none of those involved – the vaccinated and the medical staff who give them the treatment and follow them – do not know to which group they belong.

Vaccine efficacy

As early as 1915, epidemiologist Major Greenwood and statistician Edney Yule realized that a way to measure the quality of vaccines needed to be found, and first proposed a way to calculate the effectiveness of vaccines then developed against cholera and typhoid. Their method involved the use of a statistical test and more complex equations than those commonly used today.

At the beginning of the clinical study, the parameters that will be used to monitor the condition of the subjects and to determine the effectiveness of the vaccine are determined. Among other things, it is possible to examine the rate of infection in the population, how many show symptoms of the disease and how many contracted but did not become ill (asymptomatic), the proportion of patients hospitalized in intensive care units, the degree of need for respiratory support, the severity of symptoms, cost of treatment and mortality. A vaccine that improves these factors will help eradicate the epidemic.

The effect of the corona virus on those infected varies greatly from person to person. It depends on many factors, including age, sex, ethnicity and background diseases. Thus, and especially because of the burden that the disease places on the limited infrastructure of the health system, the most critical parameter examined for it is the severe symptoms. However, even vaccines that do not affect the development of the disease, but lower the rate of infection, can be very beneficial.

Controlled experiment

In the most effective experimental method, subjects are given the vaccine or placebo doses in clinical trials, and then they are all infected with the same amount of bacteria or viruses to see which one will start, how long the disease will last and what its severity will be. This is the type of experiment done on the vaccines for typhus and malaria, in which healthy volunteers agreed to be exposed to the pathogen in a controlled manner. Such an experiment makes it possible to quickly test the effectiveness of the vaccine and rule out potential inappropriate vaccines from the outset.

In practice, experiments conducted today usually avoid, for ethical reasons, infecting the subjects proactively. Instead wait patiently and follow both groups for a long time. During this time, we examine which of the participants were naturally infected with the pathogen, monitor the development of the disease in them and analyze the data. When considering efficacy based on natural infection it is of course important to remember that each patient may have been exposed to a different amount of viruses in the first place, and this too may affect the course and severity of the disease.

When it comes to diseases that are not very common and not very contagious, this stage can take years. Also, the rarer the disease the more participants in the experiment need, otherwise there will not be enough contagious among them to establish the statistical information. This of course is not the case in the current corona plague, which has claimed the lives of more than a million people in six months and infected tens of millions. In this case, it is precisely the rapid spread of the virus that helps greatly in the development of vaccines, as this is enough in a few months to accumulate many infections among the trial participants, which will provide a solid basis for statistics and the effectiveness of the vaccine.

In such a natural study, the level of efficacy of the vaccine under controlled conditions (Efficacy) is measured by the rate of decrease in the prevalence of the disease in the vaccinated group compared to the control group. That is, the less vaccinated the vaccine group is compared to the control group, the more effective the vaccine. Efficacy can be calculated against a variety of indicators, such as the rate of infection, the rate of severe symptoms and the like. For example, in order to calculate the effectiveness of the vaccine in preventing infection, the rate of infection in the control group is subtracted from the rate of infection among the vaccinated and the result is divided by the rate of infection in the control group only. In order to correctly estimate the true rate of infection in each group it is of course important to locate the asymptomatic carriers as well.

We will demonstrate this according to the vaccine data of the modern company, which was recently approved for use. In the third phase of the clinical trials, approximately 30,400 subjects participated, divided into two groups equal in size to vaccines and controls. The vaccinated group had 11 patients, compared with 185 in the control group. The difference between the morbidity in the two groups is 174, so the effectiveness of the vaccine in preventing the disease is 174/185, i.e. about 94 percent, as indeed reported. That is, there was a 94 percent decrease in the prevalence of the disease in the vaccinated compared to the unvaccinated. If the size of the groups were different, we would also have to take it into account in our calculation.

The efficacy of the vaccine under controlled conditions can also be similarly measured based on additional characteristics of the participants. Examining the effectiveness of modern vaccine by age, it is found that it stood at 95.6 percent at ages 65-18 and decreased to 86.4 percent in vaccinated people aged 65 and older.

The effectiveness of the vaccine can also be linked to the severity of the patients’ symptoms, as sometimes a vaccine does not prevent the disease, but allows the body to cope better with it and prevent dangerous deterioration. Indeed, in a modern company trial, 30 of the patients in the control group had severe symptoms of the disease, compared to zero of the true vaccine recipients. However, it is difficult to estimate in controlled trials the degree of protection that the vaccine gives against severe symptoms and death, as this requires much more subjects. It is therefore reliably tested only after the vaccine has been distributed to the general population, in the fourth phase of clinical trials or in epidemiological studies.

The price of control

Testing the effectiveness of a vaccine under controlled conditions provides an estimate of the best-case scenario regarding the protection afforded by the vaccine. Such research allows us to control the experimental conditions, make sure that the composition of the two groups will be similar and monitor the medical condition of the patients over time. The selection of participants in the experiment and their division into groups is critical, because of the differences in the risk factors and the behavior of different population groups. For example, young people attend social gatherings more than 40 years old, and therefore may be more exposed to the virus. So the match between the vaccine group and the control group is essential for the reliability of the study.

However, this advantage of the controlled studies is also a disadvantage. Most of the people selected to participate in clinical trials are physically healthy people, so the efficacy estimate obtained from the study is an overestimation that does not take into account more vulnerable populations. If the same vaccine had been tested on the elderly or only chronically ill patients, it is likely that many more of them would have suffered from severe symptoms. In such a case, an efficiency measure based solely on them would have been rather biased downwards.

Reality conditions

After the vaccine is released on the market, its effectiveness in the conditions of reality (Effectiveness) is also examined, ie the ability of the vaccine to prevent the actual results of the disease, in the complex reality. This type of efficacy refers to the total advantages and disadvantages of the complete vaccination program, and not just the vaccine itself, as they are expressed in the field, and not under controlled laboratory conditions.

The effectiveness of the vaccine under the conditions of reality is of course affected by its effectiveness under controlled conditions, but also by the rate of actual vaccinators, its distribution difficulties, the public’s access to the vaccine, geography, demographics and more. For example, the more people are vaccinated, the more the population will benefit from the effect of the herd immunity effect — that is, a condition in which the pathogen fails to spread because patients encounter only vaccinated people. This condition is not measured under laboratory conditions.

There are quite a few factors that affect the examination of the effectiveness of the vaccine in the conditions of reality. Thus, it is sometimes difficult to separate the effectiveness of the vaccine from other variables that exist in the field and also affect morbidity, such as age and background diseases.

Thus, the calculation of the effectiveness of a vaccine under the conditions of reality is usually done only retrospectively, looking back on a very large population. Due to the size of the vaccinated population it is also possible to identify very rare side effects of the vaccine, which will not be detected in clinical trials whose scope is limited. Examining the effectiveness in real-world conditions will reveal how factors such as background, sex, age and pregnancy affect the effectiveness of the vaccine, and even examine what happens with re-infection.

Optimize efficiency

To streamline and lower these advanced clinical stages, the World Health Organization has proposed creating a single placebo vaccine control group that will provide the data for comparison with the experimental groups of several different companies developing new vaccines. In the same way, we can compare the effectiveness and safety of the various vaccines. Although Pfizer and Moderna’s vaccines have already been approved, many more companies are in the process of development and will be able to use a joint control group, although this solution is not without its problems. Finally, examining the efficacy of each vaccine under controlled and realistic conditions will make it easier for us to decide which vaccine to give to each section of the population.