Medical research results: why outcome switching matters

Some researchers are doing a bait-and-switch with medical data

Medical research outcome switching is an extremely common problem that distorts the evidence doctors use to make real-world clinical decisions.

These doctors want to fix a huge problem with drug trials. Why isn’t anyone listening?, vox, February 25, 2016.

When studying a drug or device, researchers measure dozens of health outcomes to understand how the new technology works. Before they start a clinical trial, however, they’re supposed to pre-specify which outcomes they really care about on public clinical trials registries… The idea is that researchers won’t later selectively publish the most positive or more favorable outcomes and drop the negative ones.

…In the real world, unfortunately, researchers sometimes cherry-pick results, leaving doctors and patients with a skewed picture of how drugs and devices actually work…

How did NEJM respond when we tried to correct 20 misreported trials?, compare-trials, February 25, 2016.

Significant medical data

How some scientists – and news – can play with medical data…

How Scientists Are Doing A Bait-And-Switch With Medical Data

A scientific result is considered “statistically significant” if there’s a less than one in 20 chance that it could happen by chance – if you measure 20 different things, you’ll probably get one positive result, as this XKCD demonstrates.

Sources
  • Image: Significant, xkcd webcomic of romance, sarcasm, math, and language.
  • How Scientists Are Doing A Bait-And-Switch With Medical Data, BuzzFeed, Jan. 22, 2016.
  • Enjoy our health comic strips album on Flickr.
Related