The James Lind Library’s Introduction to Fair Tests of Treatments

JLL Book of Essays, About Fair Tests, May 2019


At various times in our lives and to varying levels of intensity, we all use, provide or pay for health and social care. As we decide what to do, take, offer or buy, we need evidence that is reliable, robust and trustworthy about different options. Even before James Lind’s experiment comparing possible treatments for scurvy on HMS Salisbury people had recognised that getting this evidence requires strenuous efforts to reduce bias –but that achieving this is often not straightforward. This book of essays from the James Lind Library is our attempt to illustrate some of the challenges encountered and how to overcome them.

We will take you on a journey through the sometimes stormy waters of why treatments need to be tested, rather than being based on assumptions that “it must work” before the treatment has even been tried, or based on impressions after it has been used a few times, through to the need for fair tests comparing alternative treatment options. We will show why genuine uncertainties must be identified and addressed, and how research to find the most effective and appropriate treatments need to build on research to identify the most effective and appropriate methods for doing that research. We will navigate through the reasons why comparisons need to be fair at the outset, and then kept fair as the treatments being tested are given;outcomes are measured;and results are analysed, reported, and combined in systematic reviews of all the relevant, trustworthy evidence.

We have not cluttered the chapters with references to all the source material on which we have drawn. For that level of detail, please follow the links to the fuller essays on the James Lind Library website. Instead, where we know of reviews of methodology research which are relevant to a topic, we have listed these at the end of each chapter.

By the end of the book, we hope that you will recognise how, to bring benefits of research to patients and the public, systematic reviews of fair tests are needed to provide key elements of the knowledge needed to inform decisions about health and social care, while taking into account other important factors, such as values, preferences, needs, resources and priorities. We also hope that, as you finish the book, you will share the sense of enlightenment, education and enjoyment that we have gained from preparing it.

Finally, we dedicate this book to England’s National Institute for Health Research. Without the Institute’s 16-year-long support for the James Lind Initiative, the home of the James Lind Library during that time, neither the Library nor these essays would have been possible. And we also wish to acknowledge the role the Institute plays in recognising the vital contribution of research to the delivery of health and social care that is effective and efficient, and the Institute’s leadership in ensuring that the research itself is effective, efficient and reliable, with minimal waste.


1.3 Why treatment comparisons must be fair

Untrustworthy treatment comparisons are those in which biases, or the play of chance, or both result in misleading estimates of the effects of treatments. Fair treatment comparisons avoid biases and reduce the effects of the play of chance.

It is not only failure to test theories about treatments in practice that has caused preventable tragedies. They have also occurred because the tests used to assess the effects of treatments have been unreliable and misleading. In the 1950s, theory and poorly controlled tests yielded unreliable evidence suggesting that diethylstilboestrol (DES) helped pregnant women who had previously had miscarriages and stillbirths. Although fair tests suggested that DES was useless, theory and unreliable evidence, together with aggressive marketing, led to DES being prescribed to millions of pregnant women over the next few decades. The consequences were disastrous for the women and their children, who experienced infertility and cancers as a result. The lesson is that a treatment that has not been reliably shown to be useful should not be promoted.

Problems resulting from inadequate tests of treatments continue to occur. Again, because of unreliable evidence and aggressive marketing, millions of women were persuaded to use hormone replacement therapy (HRT). It was claimed that, not only could it reduce unpleasant menopausal symptoms, but also the chances of having heart attacks and strokes. When these claims were assessed in fair tests, the results showed that in women over 60, far from reducing the risks of heart attacks and strokes, HRT increases the risks of these life-threatening conditions, as well as having other undesirable effects.These examples of the need for fair tests of treatments are a few of many that illustrate how treatments can do more harm than good. Improved general knowledge about fair tests of treatments is needed so that –laced with a healthy dose of scepticism –we can all assess claims about the effects of treatments more critically. That way, we will all become more able to judge which treatments are likely to do more good than harm.

DES DiEthylStilbestrol Resources

Nearly half of (Italian) playgrounds tested contaminated by pesticide(s)

Pesticide contamination and associated risk factors at public playgrounds near intensively managed apple and wine orchards

2019 Study Abstract

Pesticide levels are generally monitored within agricultural areas, but are commonly not assessed at public places. To assess possible contamination of non-target areas, 71 public playgrounds located next to intensively managed apple and wine orchards were selected in four valleys of South Tyrol (northern Italy). Further, the impact of environmental site characteristics on the number and concentration of pesticides was assessed. Grass samples from the selected playgrounds were collected and screened for 315 pesticide residues using standard gas chromatography and mass spectrometry.

Nearly half of the playgrounds (45%) were contaminated by at least one pesticide and a quarter (24%) by more than one. Eleven of the 12 different detected pesticides are classified as endocrine-active substances including the insecticide phosmet and the fungicide fluazinam showing the highest concentrations (0.069 and 0.26 mg kg−1, respectively). Additionally, one disinfectant and one preservation agent was found. Playgrounds in Venosta valley were most often contaminated (76% of all investigated playgrounds), highest concentrations were found in the Low Adige (2.02 mg kg−1). Pesticide concentrations were positively associated with areal proportion of apple orchards in the surroundings, the amount of rainfall and wind speed. In contrast, increasing global irradiance, opposite wind direction, increasing distance to agricultural sites and high wind speeds when pesticide application was not allowed were associated with decreasing pesticide contamination.

This study is among the first investigating pesticide contamination of public playgrounds together with environmental factors in areas with pesticide-intensive agriculture at the beginning of the growing season. It is likely that playgrounds will be affected by more pesticides and higher concentrations over the course of the crop season. The result, that the majority of the detected pesticides are classified as endocrine active is worrisome as children are especially vulnerable. Hence, we recommend that pesticide risk assessments should better include protection measures for non-target areas.
Image Antonio Thomás Koenigkam Oliveira.

Comment fonctionnent les perturbateurs endocriniens ?

Le Monde, vidéo publiée le 12 juillet 2017

Ils sont partout ! Comment perturbent-ils ?

Vidéo Le Monde, publiée le 12 juillet 2017

Le Distilbène DES, en savoir plus

DES Action Australia

DES : The wonder drug you should wonder about – In Australia

DES Action Australia formed in late 1979 and, in retrospect, it was a result of serendipity, several independent factors aligning.

Continue readingAbout DES Action Australia“.
– Watch the slideshare presentation : DES Action in Australia.
– Follow DES Action in Australia on Facebook.

DES DiEthylStilbestrol Resources

Chronic use of tramadol after acute pain episode: cohort study

Tramadol use is associated with a higher risk of prolonged opioid use in patients with an acute episode of pain compared with other short acting opioids, finds new research

2019 Study Abstract

To determine the risk of prolonged opioid use in patients receiving tramadol compared with other short acting opioids.

Observational study of administrative claims data.

United States commercial and Medicare Advantage insurance claims (OptumLabs Data Warehouse) January 1, 2009 through June 30, 2018.

Opioid-naive patients undergoing elective surgery.

Main outcome measure
Risk of persistent opioid use after discharge for patients treated with tramadol alone compared with other short acting opioids, using three commonly used definitions of prolonged opioid use from the literature: additional opioid use (defined as at least one opioid fill 90-180 days after surgery); persistent opioid use (any span of opioid use starting in the 180 days after surgery and lasting ≥90 days); and CONSORT definition (an opioid use episode starting in the 180 days after surgery that spans ≥90 days and includes either ≥10 opioid fills or ≥120 days’ supply of opioids).

Of 444 764 patients who met the inclusion criteria, 357 884 filled a discharge prescription for one or more opioids associated with one of 20 included operations. The most commonly prescribed post-surgery opioid was hydrocodone (53.0% of those filling a single opioid), followed by short acting oxycodone (37.5%) and tramadol (4.0%). The unadjusted risk of prolonged opioid use after surgery was 7.1% (n=31 431) with additional opioid use, 1.0% (n=4457) with persistent opioid use, and 0.5% (n=2027) meeting the CONSORT definition. Receipt of tramadol alone was associated with a 6% increase in the risk of additional opioid use relative to people receiving other short acting opioids (incidence rate ratio 95% confidence interval 1.00 to 1.13; risk difference 0.5 percentage points; P=0.049), 47% increase in the adjusted risk of persistent opioid use (1.25 to 1.69; 0.5 percentage points; P<0.001), and 41% increase in the adjusted risk of a CONSORT chronic opioid use episode (1.08 to 1.75; 0.2 percentage points; P=0.013).

People receiving tramadol alone after surgery had similar to somewhat higher risks of prolonged opioid use compared with those receiving other short acting opioids. Federal governing bodies should consider reclassifying tramadol, and providers should use as much caution when prescribing tramadol in the setting of acute pain as for other short acting opioids.

Chemicals leaching from plastic pollution impairs the base of the marine food web

Plastic leachates impair growth and oxygen production in Prochlorococcus, the ocean’s most abundant photosynthetic bacteria

2019 Study Abstract

Plastic pollution is a global threat to marine ecosystems. Plastic litter can leach a variety of substances into marine environments; however, virtually nothing is known regarding how this affects photosynthetic bacteria at the base of the marine food web.

To address this, we investigated the effect of plastic leachate exposure on marine Prochlorococcus, widely considered the most abundant photosynthetic organism on Earth and vital contributors to global primary production and carbon cycling.

Two strains of Prochlorococcus representing distinct ecotypes were exposed to leachate from common plastic items: high-density polyethylene bags and polyvinyl chloride matting.

We show leachate exposure strongly impairs Prochlorococcus in vitro growth and photosynthetic capacity and results in genome-wide transcriptional changes. The strains showed distinct differences in the extent and timing of their response to each leachate.

Consequently, plastic leachate exposure could influence marine Prochlorococcus community composition and potentially the broader composition and productivity of ocean phytoplankton communities.

Environmental Health Concerns From Unconventional Natural Gas Development

After a decade of research, here’s what scientists know about the health impacts of fracking

Hydraulic fracturing, also known as fracking, is a process of extracting oil and gas from the Earth by drilling deep wells and injecting a mixture of liquids and chemicals at high pressure. According to a new study, fracking has been linked to preterm births, high-risk pregnancies, asthma, migraine headaches, fatigue, nasal and sinus symptoms, and skin disorders over the last 10 years, Environmental Health News reports. Image Victoria Buchan-Dyer.

Summary and Keywords

Unconventional natural gas development (UNGD), which includes the processes of horizontal drilling and hydraulic fracturing to extract natural gas from unconventional reservoirs such as shale, has dramatically expanded since 2000. In parallel, concern over environmental and community impacts has increased along with the threats they pose for health. Shale gas reservoirs are present on all continents, but only a small proportion of global reserves has been extracted through 2016. Natural gas production from UNGD is highest in the United States in Pennsylvania, Texas, Louisiana, Oklahoma, and Arkansas. But unconventional production is also in practice elsewhere, including in eighteen other U.S. states, Canada, and China. Given the rapid development of the industry coupled with its likelihood of further growth and public concern about potential cumulative and long-term environmental and health impacts, it is important to review what is currently known about these topics.

The environmental impacts from UNGD include chemical, physical, and psychosocial hazards as well as more general community impacts. Chemical hazards commonly include detection of chemical odors; volatile organic compounds (including BTEX chemicals [benzene, toluene, ethylbenzene, and xylene], and several that have been implicated in endocrine disruption) in air, soil, and surface and groundwater; particulate matter, ozone, and oxides of nitrogen (NOx) in air; and inorganic compounds, including heavy metals, in soil and water, particularly near wastewater disposal sites. Physical hazards include noise, light, vibration, and ionizing radiation (including technologically enhanced naturally occurring radioactive materials [TENORMs] in air and water), which can affect health directly or through stress pathways. Psychosocial hazards can also operate through stress pathways and include exposure to increases in traffic accidents, heavy truck traffic, transient workforces, rapid industrialization of previously rural areas, increased crime rates, and changes in employment opportunities as well as land and home values. In addition, the deep-well injection of wastewater from UNGD has been associated with increased seismic activity.

These environmental and community impacts have generated considerable concern about potential health effects and corresponding political debate over whether UNGD should be promoted, regulated, or banned. For several years after the expansion of the industry, there were no well-designed, population-based studies that objectively measured UNGD activity or associated exposures in relation to health outcomes. This delay is inherent after the introduction of new industries, but hundreds of thousands of wells were drilled before any health studies were completed. By 2017, there were a number of important, peer-reviewed studies published in the scientific literature that raised concern about potential ongoing health impacts. These studies have reported associations between proximity to UNGD and pregnancy and birth outcomes; migraine headache, chronic rhinosinusitis, severe fatigue, and other symptoms; asthma exacerbations; and psychological and stress-related concerns. Beyond its direct health impacts, UNGD may be substantially contributing to climate change (due to fugitive emissions of methane, a powerful greenhouse gas), which has further health impacts. Certain health outcomes, such as cancer and neurodegenerative diseases, cannot yet be studied because insufficient time has passed in most regions since the expansion of UNGD to allow for latency considerations. With the potential for tens of thousands of additional wells across large geographic areas, these early health studies should give pause about whether and how UNGD should proceed. Citing health concerns, several U.S. states and nations in Europe have already decided to not allow UNGD.

The Primodos Issue ; debated by Lord Alton of Liverpool

UK Parliament, House of Lords Hansard, 28 February 2019

“They have faced the implacable determination of regulatory bodies spending huge amounts of public money on ad hoc scientific reviews to cast doubt on the work of highly reputable scientists. Those who have suffered so grievously deserve much better than this.”

Safety of Medicines and Medical Devices, House of Lords Hansard Debate, 28 February 2019.

More information

Primodos, Sodium Valproate, Surgical Mesh : Baroness Cumberlege Talks

UK Parliament, House of Lords Hansard, 28 February 2019

“For the families involved, it is life-changing and extremely distressing. For those women who took Primodos and sodium valproate, there is an intense feeling of guilt. They took the medication and they blame themselves. However hard one tries to persuade them that it was not their fault, the guilt remains.”

“That tells me something is seriously wrong; the system is not working as it should. People who have been harmed should not have to fight to be heard or to access the care they need.”

More information

Lords debate Safety of Medicines and Medical Devices

UK Parliament, House of Lords Hansard, 28 February 2019

Lords debates medicinal safety in ref to the public health scandals involving the hormone-based pregnancy test drug Primodos, the use of vaginal mesh implants and the anti-epilepsy drug sodium valproate.

  • Interventions from Lord O’Shaughnessy, Lord Hunt of Kings Heath, Baroness Walmsley, Baroness Masham of Ilton, Baroness Cumberlege 36:28 , Lord Brennan, Lord Carrington, Lord Bethell, Baroness Bryan of Partick, Lord Alton of Liverpool 1:17:29 , Lord Suri, Baroness Finlay of Llandaff, The Earl of Dundee, Baroness Jolly, Baroness Thornton, The Parliamentary Under-Secretary of State, Department of Health and Social Care (Baroness Blackwood of North Oxford).
  • Read the Lords Hansard transcript.
  • Parliamentary news, research briefing.
  • Parliament news, press release.
  • Video source, Parliament Tv.

More information