Experimental Studies Without a Real Control Group

In the previous parts we saw that, to measure the effect of a treatment, two groups of patients are compared, and that this comparison only means something when the two groups are alike in every respect other than the treatment itself. An RCT builds this likeness through randomization, and through blinding it prevents knowledge of the group from affecting the behaviour of patient and researcher after the study begins. In this part we reach two kinds of experimental study that lack these components. In one there is no control group at all; in the other there is a control group, but it is built from data from previous years. Both are seen often in the dental literature and both count as weak evidence, but the reason for the weakness differs in each.

The study without a control group

The simplest form of experimental study is for the researcher to give one treatment to all patients and record the result. This design is called an uncontrolled study. For example (a hypothetical example), twenty patients are treated with a new type of implant, and after one year it is reported that eighteen of them had no problem at all. In this study there is no group treated with the usual implant against which this result could be compared. Without such a comparison, the number eighteen on its own says nothing about whether the new implant is better or worse, because it is not known how many of these same twenty people would have had a problem had they been treated with the usual implant. Perhaps the same eighteen, perhaps more.

For this reason an uncontrolled study is very weak evidence, and clinical decisions should not be based on it. But that does not mean it is useless. The right place for this study is at the start of work on a new treatment, that is, before the main study has been designed. At this stage the researcher has a few simple questions: Is this treatment safe at all? Does it have a complication we did not expect? Roughly what range does its result fall in? The answers help them plan the larger, more precise study properly. The problem begins when the result of this preliminary study, instead of being the prelude to the main study, is itself used as the final answer.

The historical control group

One step up from the uncontrolled study is a study that does have a control group, but one that was not treated at the same time as the treatment group; instead it has been drawn from past records. For example (a hypothetical example), patients treated today with a new technique are compared with the records of patients treated five years earlier at the same centre with the old technique. Sometimes, instead of records, the results of an older study are used as the comparison group. This group is called a historical control, meaning a comparison group built from data from a different time. This design, too, is usually used at the same preliminary stage as the uncontrolled study, with the difference that there is at least something to compare against.

Even so, this design is also weak in two respects. The first concerns the time gap. In the five years separating those patients from today's patients, a great deal may have changed: materials, equipment, the skill of the treatment team, the kind of patients who attend that centre, even the diagnostic criteria. So if the new technique shows a better result, we do not know whether the difference is due to the technique itself or to everything else that improved in the meantime. The second concerns the quality of the control group's data. Those records were written by someone else, at another time, and for the purpose of treatment, not research. There is no guarantee that bias did not enter when they were collected or recorded, or later when they were read again for the study. This is the same problem of retrospective data we saw in part one, with the difference that here only one of the two groups suffers from it.

Summary

Both designs in this part share one shortcoming: neither has a control group built and followed at the same time as the treatment group. In the uncontrolled study there is no comparison group at all, and with a historical control there is a group, but it comes from another time. Both are useful for starting work on a new treatment, and both become misleading when used in place of final evidence.

Everything we have seen so far has been experimental, meaning the researcher decided who received which treatment. From the next part we enter the observational family, where the groups are formed by real life rather than by the researcher. The first and most important example of this family is the cohort.