The James Lind Library’s Introduction to Fair Tests of Treatments

JLL Book of Essays, About Fair Tests, May 2019

Introduction

At various times in our lives and to varying levels of intensity, we all use, provide or pay for health and social care. As we decide what to do, take, offer or buy, we need evidence that is reliable, robust and trustworthy about different options. Even before James Lind’s experiment comparing possible treatments for scurvy on HMS Salisbury people had recognised that getting this evidence requires strenuous efforts to reduce bias –but that achieving this is often not straightforward. This book of essays from the James Lind Library is our attempt to illustrate some of the challenges encountered and how to overcome them.

We will take you on a journey through the sometimes stormy waters of why treatments need to be tested, rather than being based on assumptions that “it must work” before the treatment has even been tried, or based on impressions after it has been used a few times, through to the need for fair tests comparing alternative treatment options. We will show why genuine uncertainties must be identified and addressed, and how research to find the most effective and appropriate treatments need to build on research to identify the most effective and appropriate methods for doing that research. We will navigate through the reasons why comparisons need to be fair at the outset, and then kept fair as the treatments being tested are given;outcomes are measured;and results are analysed, reported, and combined in systematic reviews of all the relevant, trustworthy evidence.

We have not cluttered the chapters with references to all the source material on which we have drawn. For that level of detail, please follow the links to the fuller essays on the James Lind Library website. Instead, where we know of reviews of methodology research which are relevant to a topic, we have listed these at the end of each chapter.

By the end of the book, we hope that you will recognise how, to bring benefits of research to patients and the public, systematic reviews of fair tests are needed to provide key elements of the knowledge needed to inform decisions about health and social care, while taking into account other important factors, such as values, preferences, needs, resources and priorities. We also hope that, as you finish the book, you will share the sense of enlightenment, education and enjoyment that we have gained from preparing it.

Finally, we dedicate this book to England’s National Institute for Health Research. Without the Institute’s 16-year-long support for the James Lind Initiative, the home of the James Lind Library during that time, neither the Library nor these essays would have been possible. And we also wish to acknowledge the role the Institute plays in recognising the vital contribution of research to the delivery of health and social care that is effective and efficient, and the Institute’s leadership in ensuring that the research itself is effective, efficient and reliable, with minimal waste.

Abstract

1.3 Why treatment comparisons must be fair

Untrustworthy treatment comparisons are those in which biases, or the play of chance, or both result in misleading estimates of the effects of treatments. Fair treatment comparisons avoid biases and reduce the effects of the play of chance.

It is not only failure to test theories about treatments in practice that has caused preventable tragedies. They have also occurred because the tests used to assess the effects of treatments have been unreliable and misleading. In the 1950s, theory and poorly controlled tests yielded unreliable evidence suggesting that diethylstilboestrol (DES) helped pregnant women who had previously had miscarriages and stillbirths. Although fair tests suggested that DES was useless, theory and unreliable evidence, together with aggressive marketing, led to DES being prescribed to millions of pregnant women over the next few decades. The consequences were disastrous for the women and their children, who experienced infertility and cancers as a result. The lesson is that a treatment that has not been reliably shown to be useful should not be promoted.

Problems resulting from inadequate tests of treatments continue to occur. Again, because of unreliable evidence and aggressive marketing, millions of women were persuaded to use hormone replacement therapy (HRT). It was claimed that, not only could it reduce unpleasant menopausal symptoms, but also the chances of having heart attacks and strokes. When these claims were assessed in fair tests, the results showed that in women over 60, far from reducing the risks of heart attacks and strokes, HRT increases the risks of these life-threatening conditions, as well as having other undesirable effects.These examples of the need for fair tests of treatments are a few of many that illustrate how treatments can do more harm than good. Improved general knowledge about fair tests of treatments is needed so that –laced with a healthy dose of scepticism –we can all assess claims about the effects of treatments more critically. That way, we will all become more able to judge which treatments are likely to do more good than harm.

DES DiEthylStilbestrol Resources

About perimenopausal and postmenopausal women using hormones treatment

Barbara Seaman, New York University, 2003

Barbara Seaman, American author, activist, and journalist, talks.

It’s My Ovaries, Stupid !

Dr. Vliet offers a different perspective on hormones

This landmark work in women’s health identifies and offers solutions to the hormonal dysfunctions afflicting millions of young women, teens, and even children, that rob women of future fertility and contribute to devastating problems — from early onset puberty and obesity to depression and increased cancer risk. Women’s health is more than breast cancer, pregnancy, and menopause. In this groundbreaking new work, Dr. Elizabeth Lee Vliet identifies and explains rarely acknowledged, pervasive threats to young women’s health and fertility — PCOS (Polycystic Ovary Syndrome), POD (Premature Ovarian Decline), and Premature Ovarian Failure (menopause in the young) — and the overlooked causes of endometriosis, cystitis, early puberty, allergies, heart disease, mood disorders, depression, chronic fatigue, fibromyalgia, bone loss, anxiety, obesity, and diabetes.

A kind of “Silent Spring” of women’s health, “It’s My Ovaries, Stupid!” presents compelling evidence from worldwide research that common environmental toxins and endocrine disruptors in pesticides, plastic food wrappers, food additives, preservatives, soy supplements, aspartame in diet sodas and junk food, and more — as well as lifestyle factors such as stress — can all profoundly disrupt hormone function, even in childhood.Insidious robbers of quality of life, fertility, and health, hormone dysfunctions are on the rise today, afflicting younger and younger women.Why? What can you do about it? How can you get tested? What treatments are available? Dr. Vliet interprets the latest scientific research and draws on more than twenty years of clinical experience to answer these and many other crucial questions about commonhealth problems in young women.Whose job is it to take care of the ovaries…beyond their function in reproduction? Why do you have trouble getting help for “hormone problems” that are clearly linked to your monthly cycle?

It’s My Ovaries, Stupid!” bridges this gap in women’s health care and shows you how to understand your symptoms and get reliable tests, how to receive treatment and improve your health, how to wade through the controversies surrounding hormone replacement therapy, and how to explore cutting-edge options for thyroid problems.You can’t afford “not” to read this book. Your life, your fertility, and your long-term health may depend on it. It’s not all in your head, and it’s not just stress. It’s your ovaries!

More Information
Endocrine Disruptors

A Study of High Frequency Editorialists Favoring Hormone Replacement Therapy

Partisan Perspectives in the Medical Literature, Journal of General Internal Medicine, 2010

2010 Study Abstract

Background
Unfavorable results of major studies have led to a large shrinkage of the market for hormone replacement therapy (HRT) in the last 6 years. Some scientists continue to strongly support the use of HRT.

Objectives
We analyzed a sample of partisan editorializing articles on HRT to examine their arguments, the reporting of competing interests, the journal venues and their sponsoring societies.

Data Sources
Through Thomson ISI database, we selected articles without primary data written by the five most prolific editorialists that addressed clinical topics pertaining to HRT and that were published in regular journal issues in 2002–2008.

Main Measures
We recorded the number of articles with a partisan stance and their arguments, the number of partisan articles that reported conflicting interests, and the journal venues and their sponsoring societies publishing the partisan editorials.

Key Results
We analyzed 114 eligible articles (58 editorials, 16 guidelines, 37 reviews, 3 letters), of which 110 (96%) had a partisan stance favoring HRT. Typical arguments were benefits for menopausal and related symptoms (64.9%), criticism of unfavorable studies (78.9%), preclinical data that showed favorable effects of HRT (50%), and benefits for major outcomes such as osteoporosis and fractures (49.1%), cardiovascular disease (31.6%), dementia (24.6%) or colorectal cancer (20.2%), but also even breast cancer (4.4%). All 5 prolific editorialists had financial relationships with hormone manufacturers, but these were reported in only 6 of the 110 partisan articles. Four journals published 15–37 partisan articles each. The medical societies of these journals reported on their websites that several pharmaceutical companies sponsored them or their conferences.

Conclusions
There is a considerable body of editorializing articles favoring HRT use and very few of these articles report conflicts of interest. Full disclosure of conflicts of interest is needed, especially for articles without primary data.

Sources and Press Releases
  • Partisan Perspectives in the Medical Literature: A Study of High Frequency Editorialists Favoring Hormone Replacement Therapy, Journal of General Internal Medicine, September 2010, Volume 25, Issue 9, pp 914–919, 2010.
  • More naming and shaming, ripe-tomato, APRIL 17, 2012.
  • Editorials or review authors with financial ties to hormone therapy (HT) manufacturers, in which there appeared to be bias in favour of HT featured image credit Jim Thornton.

Sex Hormones and Related Compounds, Including Hormonal Contraceptives

ResearchGate, Side Effects of Drugs Annual, July 2017

Abstract

This is a review of publications from January 2016 to December 2016 on sex hormones and related compounds. This chapter covers estrogens (diethylstilbestrol, estradiol and derivatives), progestins (drospirenone, levonorgestrel, medroxyprogesterone, ulipristal), hormone replacement therapy (combination estrogen and progestin, estrogen only, tibolone, oral preparations and topical preparations), hormonal contraceptives (oral, non-oral, combination and progestin only), in vitro fertilization agents, triptorelin (gonadotropin-releasing hormone agonist), anastrozole (aromatase inhibitor) testosterone, anabolic steroids and androgen deprivation therapy.

Request the full-text on ResearchGate.

DES DiEthylStilbestrol Resources

Can hormone replacement therapy increase the risk of hearing loss ?

The link between menopausal age, the use of oral hormonal therapy, and hearing loss

New research suggests that hormone therapy increases the risk of hearing loss among menopausal and postmenopausal women.

2017 Study Abstract

Objective

Menopause and postmenopausal hormone therapy and risk of hearing loss, menopause journal, doi: 10.1097/GME.0000000000000878, May 8, 2017.

medical news today,  articles/317387, May 10, 2017.

Image credit anthonymedicalcare.

Menopause may be a risk factor for hearing loss, and postmenopausal hormone therapy (HT) has been proposed to slow hearing decline; however, there are no large prospective studies. We prospectively examined the independent relations between menopause and postmenopausal HT and risk of self-reported hearing loss.

Methods
Prospective cohort study among 80,972 women in the Nurses’ Health Study II, baseline age 27 to 44 years, followed from 1991 to 2013. Baseline and updated information was obtained from detailed validated biennial questionnaires. Cox proportional-hazards regression models were used to examine independent associations between menopausal status and postmenopausal HT and risk of hearing loss.

Results
After 1,410,928 person-years of follow-up, 18,558 cases of hearing loss were reported. There was no significant overall association between menopausal status, natural or surgical, and risk of hearing loss. Older age at natural menopause was associated with higher risk. The multivariable-adjusted relative risk of hearing loss among women who underwent natural menopause at age 50+ years compared with those aged less than 50 years was 1.10 (95% confidence interval [CI] 1.03, 1.17). Among postmenopausal women, oral HT (estrogen therapy or estrogen plus progestogen therapy) was associated with higher risk of hearing loss, and longer duration of use was associated with higher risk (P trend < 0.001). Compared with women who never used HT, the multivariable-adjusted relative risk of hearing loss among women who used oral HT for 5 to 9.9 years was 1.15 (95% CI 1.06, 1.24) and for 10+ years was 1.21 (95% CI 1.07, 1.37).

Conclusions
Older age at menopause and longer duration of postmenopausal HT are associated with higher risk of hearing loss.

Perinatal Exposure to DES Increases the Susceptibility to Develop Mammary Gland Lesions after Estrogen Replacement Therapy

Hormones and Cancer, April 2017, Volume 8, Issue 2, pp 78–89

2017 Study Abstract

The development of the mammary gland is a hormone-regulated event.

Several factors can dysregulate its growth and make the gland more susceptible to cellular transformation.

Among these factors, perinatal exposure to xenoestrogens and hormone replacement therapy has been associated with increased risk of developing breast cancer.

Perinatal Exposure to Bisphenol A or Diethylstilbestrol Increases the Susceptibility to Develop Mammary Gland Lesions After Estrogen Replacement Therapy in Middle-Aged Rats, US National Library of Medicine National Institutes of Health, Hormones and Cancer, April 2017, Volume 8, Issue 2, pp 78–89, NCBI PubMed PMID: 28078498, 2017 Apr 8.

Here, we assessed the effects induced by estrogen replacement therapy (ERT) in ovariectomized (OVX) middle-aged rats and whether perinatal exposure to diethylstilbestrol (DES) or bisphenol A (BPA) modified these effects in the mammary gland.

Pregnant rats were orally exposed to vehicle, 5 μg DES/kg/day, or 0.5 or 50 μg BPA/kg/day from gestational day 9 until weaning. Then, 12-month-old offspring were OVX and treated with 17β-estradiol for 3 months.

Morphological changes and the percentage of epithelial cells that proliferated or expressed estrogen receptor alpha (ESR1) and progesterone receptor (PR) were analyzed in mammary gland samples of 15-month-old animals.

ERT induced lobuloalveolar hyperplasia and ductal cysts in the mammary gland of middle-aged rats, associated with a higher proliferation index of epithelial cells.

Perinatal exposure to DES followed by ERT increased the number of cysts and induced the formation of fibroadenoma and ductal carcinoma in situ, without modifying the expression of ESR1 or PR.

Also, after 3 months of ERT, BPA-exposed rats had a higher incidence of ductal hyperplasia and atypical lobular hyperplasia than animals under ERT alone.

In conclusion, perinatal exposure to xenoestrogens increases the susceptibility of the mammary gland to develop cysts and hyperplastic lesions when confronted with ERT later in life.

More DES DiEthylStilbestrol Resources

How did Doctors Not Know about the Risks of Hormone Therapy?

Via NutritionFacts.org, Dr. Greger, 21 September 2016

Read our posts tagged HRT, watch our video playlist on meds: prescription drugs side effects.

Menopausal hormone therapy and breast cancer: the true size of the increased risk

British study finds combined HRT nearly triples risk of breast cancer

2% of women monitored for six years got breast cancer – and they were 2.7 times more likely to contract it if they were on combined HRT than if they were not.

Aaccording to the study’s findings, combined HRT increases the risk of breast cancer by 170%.

Abstract

Background
Menopausal hormone therapy (MHT) increases breast cancer risk; however, most cohort studies omit MHT use after enrolment and many infer menopausal age.

Methods
We used information from serial questionnaires from the UK Generations Study cohort to estimate hazard ratios (HRs) for breast cancer among post-menopausal women with known menopausal age, and examined biases induced when not updating data on MHT use and including women with inferred menopausal age.

Menopausal hormone therapy and breast cancer: what is the true size of the increased risk?, nature, 28 July 2016.

Results
Among women recruited in 2003–2009, at 6 years of follow-up, 58 148 had reached menopause and 96% had completed a follow-up questionnaire. Among 39 183 women with known menopausal age, 775 developed breast cancer, and the HR in relation to current oestrogen plus progestogen MHT use (based on 52 current oestrogen plus progestogen MHT users in breast cancer cases) relative to those with no previous MHT use was 2.74 (95% confidence interval (CI): 2.05–3.65) for a median duration of 5.4 years of current use, reaching 3.27 (95% CI: 1.53–6.99) at 15+ years of use. The excess HR was underestimated by 53% if oestrogen plus progestogen MHT use was not updated after recruitment, 13% if women with uncertain menopausal age were included, and 59% if both applied. The HR for oestrogen-only MHT was not increased (HR=1.00; 95% CI: 0.66–1.54).

British study finds combined HRT nearly triples risk of breast cancer, the guardian, 23 August 2016.

Image The Lucky Tourist.

Conclusions
Lack of updating MHT status through follow-up and inclusion of women with inferred menopausal age is likely to result in substantial underestimation of the excess relative risks for oestrogen plus progestogen MHT use in studies with long follow-up, limited updating of exposures, and changing or short durations of use.

How much of a woman’s risk of developing breast cancer might be in her control?

There are things women can do to reduce breast cancer risk

Researchers estimate that close to 30% of all breast cancers in the U.S. could be prevented if women maintained a healthy weight, do not use hormone therapy for menopause, and cut back on drinking and smoking. That conclusion held true even for women who were at a high risk for breast cancer due to family history and genetics.

Abstract

Breast Cancer Risk From Modifiable and Nonmodifiable Risk Factors Among White Women in the United States, JAMA Oncology, May 26, 2016.

Importance
An improved model for risk stratification can be useful for guiding public health strategies of breast cancer prevention.

Objective
To evaluate combined risk stratification utility of common low penetrant single nucleotide polymorphisms (SNPs) and epidemiologic risk factors.

Design, Setting, and Participants
Using a total of 17 171 cases and 19 862 controls sampled from the Breast and Prostate Cancer Cohort Consortium (BPC3) and 5879 women participating in the 2010 National Health Interview Survey, a model for predicting absolute risk of breast cancer was developed combining information on individual level data on epidemiologic risk factors and 24 genotyped SNPs from prospective cohort studies, published estimate of odds ratios for 68 additional SNPs, population incidence rate from the National Cancer Institute-Surveillance, Epidemiology, and End Results Program cancer registry and data on risk factor distribution from nationally representative health survey. The model is used to project the distribution of absolute risk for the population of white women in the United States after adjustment for competing cause of mortality.

Exposures
Single nucleotide polymorphisms, family history, anthropometric factors, menstrual and/or reproductive factors, and lifestyle factors.

Main Outcomes and Measures
Degree of stratification of absolute risk owing to

  1. nonmodifiable factors:
    • SNPs,
    • family
    • history,
    • height,
    • and some components of menstrual and/or reproductive history
  2. and modifiable factors:
    • body mass index [BMI; calculated as weight in kilograms divided by height in meters squared],
    • menopausal hormone therapy [MHT],
    • alcohol,
    • and smoking.

Results
The average absolute risk for a 30-year-old white woman in the United States developing invasive breast cancer by age 80 years is 11.3%. A model that includes all risk factors provided a range of average absolute risk from 4.4% to 23.5% for women in the bottom and top deciles of the risk distribution, respectively. For women who were at the lowest and highest deciles of nonmodifiable risks, the 5th and 95th percentile range of the risk distribution associated with 4 modifiable factors was 2.9% to 5.0% and 15.5% to 25.0%, respectively. For women in the highest decile of risk owing to nonmodifiable factors, those who had low BMI, did not drink or smoke, and did not use MHT had risks comparable to an average woman in the general population.

Here Are The Things Women Can Do to Avoid Breast Cancer, time, May 26, 2016.

Conclusions and Relevance
This model for absolute risk of breast cancer including SNPs can provide stratification for the population of white women in the United States. The model can also identify subsets of the population at an elevated risk that would benefit most from risk-reduction strategies based on altering modifiable factors. The effectiveness of this model for individual risk communication needs further investigation.