|
Cats and science together on the Internet |
The Internet hosts a wealth of information, and, for
those concerned with their health and well-being, this can be both a blessing
and a curse. I’ve recently come across a website that promotes various health
and nutrition measures under the label of “science” (and even “real good
science”!) all the while discrediting the work of some academics. While it’s obviously not the first time I’ve seen this
sort of thing, this particular website hit home because it targets a community
that’s close to me. The truth is, you can find science to support nearly
anything, but not all science is created equal. I can’t prevent people
from using “science” to support their claims, but I *can* tell you about what
actually is “real good science”, how to spot it, and how to make informed
decisions.
For this first post in the series, I want to introduce you
to the gold standard for scientifically proving that something works (for
example, that a drug treats a disease, or that a diet makes you lose weight):
the double-blind, randomized controlled
trial. Let’s start at the end:
A controlled
trial means that the intervention you are testing (for example, a pill to treat
stomach ulcers) is compared with a control intervention. The control can be a
placebo (a fake intervention), like a sugar pill. You might think that anything
works better than a sugar pill. Not so! Placebos are very effective for many
conditions (for more on placebos, stay tuned for Part 2!). That’s why it’s very
important to make sure that the effectiveness of your pill for stomach ulcers
is not due to the placebo effect.
While placebos can be the ideal control, it’s not always
practical, or ethical, to conduct a trial using a placebo. If for example you
are testing a new cancer drug, you don’t necessarily want your control group to receive a placebo, and so go drug free, for the duration of the trial.
Another type of control would be to use a drug that is already on the market, has
known effects, and has already been tested thoroughly.
Randomization in the context of a controlled clinical trial means assigning
participants randomly to either the new intervention group or the control
group. Why is randomization important? It helps avoid a phenomenon called bias.
Imagine that researchers really believe the new pill for stomach ulcers will
work better than anything that already exists. They might be tempted to assign
the sickest participants to the intervention group and the others to the
control group. Seems like the right thing to do, yes? Unfortunately, not so.
It’s possible that as stomach ulcers progress, they respond differently to
various drugs, and by not assigning the participants randomly you might mask or
exaggerate the effects of your new product. Well-designed trials should be
randomized whenever possible.
In a double-blind
randomized controlled trial, two groups of people are “blinded”. The
participants are blinded in that they don’t know whether they are receiving the
new intervention or the control, and the experimenters are blinded in that when
they are analyzing the data, they don’t know who received what. Blinding a study really helps to limit the biases. That said,
it’s not always possible to blind everyone. Sometimes an intervention (like an
exercise program) is pretty obvious. But like randomization, it should be done
whenever possible.
Double-blind, randomized controlled trial are pretty much as
good as it gets when you’re trying to prove something – they
are the most
reliable form of scientific evidence because they have all the possible
elements in place to avoid false cause-and-effect evidence. Let’s look at one last example. Say
I’m trying to prove that a special diet involving eating large quantities of
bacon helps people lose weight. So I recruit 10 overweight participants and I
closely monitor their diet, making sure they eat their extra-large amounts of bacon.
At the end of my study, I find that 6 out of 10 participants lost weight. What
does this mean? Pick the right answer:
a) Bacon is an effective weight-loss tool (60%
success! Is
that a lot?) b) Bacon didn’t change anything – had the
participants just
gone on with their regular diet, 6 of them would have lost
weight anyway c) Bacon worked as a placebo for 6 of the
participants d) The experience of being monitored closely by
researchers
led the participants to pay closer attention to their diet and
exercise, and 6 of them lost weight because aside from
bacon they improved
their diet e) Bacon worked as a weight-loss tool for 2
participants and
4 participants experienced the placebo effect f) The researchers secretly worked for the Bacon
Consortium and this whole thing was a marketing exercise g) We have no idea what this means because the
experiment was not designed well.
That’s right.
Double-blind randomized controlled trials are not entirely
without flaws (no scientific method is!). They typically take a very long time
to come together and are very expensive to conduct, which can sometimes (but
not always) mean that relatively wealthy organizations pay for them (for more
on who pays for science, stay tuned for a later installment in this series). Other flaws are shared with many different types of
research. That said, while these trials are not perfect, they are still the
gold standard. So the first thing to look for when you are researching a health
intervention online is whether a double-blind, randomized controlled trial has
been done.
Stay tuned for part 2, on the placebo effect!