Tag: Clinical trials

Spin in Published Biomedical Literature : A Systematic Review

Medical research publications : how to modify (abstracts) perception of results

Quinn Grundy presents original research that explores the nature and prevalence of spin in the biomedical literature. Video published on 11 Oct 2017. Reference.

Objective
To explore the nature and prevalence of spin in the biomedical literature.

Design
In a systematic review and meta-analysis, we searched MEDLINE, PreMEDLINE, Embase, Scopus, and handsearched reference lists for all articles published between 1946 and 24 November 2016 that included the quantitative measurement of spin in the biomedical literature for at least 1 outcome. Two independent coders extracted data on the characteristics of articles and included studies, methods for assessing spin, and all spin-related results. The data were heterogeneous; results were grouped inductively into outcome-related categories. We had sufficient data to use meta-analysis to analyze the association of industry sponsorship of research with the presence of spin.

Results
We identified 4219 articles after removing duplicates and included 35 articles that investigated spin: clinical trials (23/35, 66%); observational studies (7/35, 20%); diagnostic accuracy studies (2/35, 6%); and systematic reviews and meta-analyses (4/35, 11%), with some articles including multiple study designs. The nature and manifestations of spin varied according to study design. We grouped results into the following categories: prevalence of spin, level of spin, factors associated with spin, and effects of spin on readers’ interpretations. The highest, but also greatest variability in the prevalence of spin was present in trials (median, 57% of main texts containing spin; range, 19%-100% across 16 articles). Source of funding was hypothesized to be a factor associated with spin; however, the meta-analysis found no significant association, possibly owing to the heterogeneity of the 7 included articles.

Conclusions
Spin appears to be common in the biomedical literature, though this varies by study design, with the highest rates found in clinical trials. Spin manifests in diverse ways, which challenged investigators attempting to systematically identify and document instances of spin. Widening the investigation of factors contributing to spin from characteristics of individual authors or studies to the cultures and structures of research that may incentivize or deincentivize spin, would be instructive in developing strategies to mitigate its occurrence. Further research is also needed to assess the impact of spin on readers’ decision making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research to ensure accurate interpretation and dissemination of research.

Identification of Spin in Clinical Studies Evaluating Biomarkers in Ovarian Cancer

Medical research publications : how to modify (abstracts) perception of results

Mona Ghannad presents a systematic review which documents and classifies spin or overinterpretation, as well as facilitators of spin, in recent clinical studies evaluating performance of biomarkers in ovarian cancer. Video published on 11 Oct 2017. Reference.

Objective
The objective of this systematic review was to document and classify spin or overinterpretation, as well as facilitators of spin, in recent clinical studies evaluating performance of biomarkers in ovarian cancer.

Design
We searched PubMed systematically for all studies published in 2015. Studies eligible for inclusion described 1 or more trial designs for identification and/or validation of prognostic, predictive, or diagnostic biomarkers in ovarian cancer. Reviews, animal studies, and cell line studies were excluded. All studies were screened by 2 reviewers. To document and characterize spin, we collected information on the quality of evidence supporting the study conclusions, linking the performance of the marker to outcomes claimed.

Results
In total, 1026 potentially eligible articles were retrieved by our search strategy, and 345 studies met all eligibility criteria and were included. The first 200 studies, when ranked according to publication date, will be included in our final analysis. Data extraction was done by one researcher and validated by a second. Specific information extracted and analyzed on study and journal characteristics, key information on the relevant evidence in methods, and reporting of conclusions claimed for the first 50 studies is provided here. Actual forms of spin and facilitators of spin were identified in studies trying to establish the performance of the discovered biomarker.

Actual forms of spin identified as shown (Table) were:

  1. other purposes of biomarker claimed not investigated (18 of 50 studies [36%]);
  2. incorrect presentation of results (15 of 50 studies [30%]);
  3. mismatch between the biomarker’s intended clinical application and population recruited (11 of 50 studies [22%]);
  4. mismatch between intended aim and conclusion (7 of 50 studies [14%]);
  5. and mismatch between abstract conclusion and results presented in the main text (6 of 50 studies [12%]).

Frequently observed facilitators of spin were:

  1. not clearly prespecifying a formal test of hypothesis (50 of 50 studies [100%]);
  2. not stating sample size calculations (50 of 50 studies [100%]);
  3. not prespecifying a positivity threshold of continuous biomarker (17 of 43 studies [40%]);
  4. not reporting imprecision or statistical test for data shown (ie, confidence intervals, P values) (12 of 50 studies [24%]);
  5. and selective reporting of significant findings between results for primary outcome reported in abstract and results reported in main text (9 of 50 studies [18%]).

Conclusions
Spin was frequently documented in abstracts, results, and conclusions of clinical studies evaluating performance of biomarkers in ovarian cancer. Inflated and selective reporting of biomarker performance may account for a considerable amount of waste in the biomarker discovery process. Strategies to curb exaggerated reporting are needed to improve the quality and credibility of published biomarker studies.

Angell on Big Pharma

Some more truth about the pharmaceutical companies

Do pharmaceutical companies corrupt academic research and the clinical trial process ? You bet.

Dr Marcia Angell of Harvard Medical School and the author of The Truth About the Drug Companies talks with EconTalk host Russ Roberts about the impact of pharmaceutical companies on academic research, clinical trials and the political process. Angell argues that the large pharmaceutical companies produce little or no innovation and use their political power to exploit consumers and taxpayers. Reference.

Medical research publications : how to modify (abstracts) perception of negative (or non-significant) results into positive ones

Evaluation of Spin in the Abstracts of Emergency Medicine Randomized Controlled Trials

Spin is common (>40%) in emergency medicine randomized controlled trials…

May 2019 Study objective

We aim to investigate spin in emergency medicine abstracts, using a sample of randomized controlled trials from high-impact-factor journals with statistically nonsignificant primary endpoints.

Methods

This study investigated spin in abstracts of emergency medicine randomized controlled trials from emergency medicine literature, with studies from 2013 to 2017 from the top 5 emergency medicine journals and general medical journals. Investigators screened records for inclusion and extracted data for spin. We considered evidence of spin if trial authors focused on statistically significant results, interpreted statistically nonsignificant results as equivalent or noninferior, used favorable rhetoric in the interpretation of nonsignificant results, or claimed benefit of an intervention despite statistically nonsignificant results.

Results

Of 772 abstracts screened, 114 randomized controlled trials reported statistically nonsignificant primary endpoints. Spin was found in 50 of 114 abstracts (44.3%). Industry-funded trials were more likely to have evidence of spin in the abstract (unadjusted odds ratio 3.4; 95% confidence interval 1.1 to 11.9). In the abstracts’ results, evidence of spin was most often due to authors’ emphasizing a statistically significant subgroup analysis (n=9). In the abstracts’ conclusions, spin was most often due to authors’ claiming they accomplished an objective that was not a prespecified endpoint (n=14).

Conclusion

Spin was prevalent in the selected randomized controlled trial, emergency medicine abstracts. Authors most commonly incorporated spin into their reports by focusing on statistically significant results for secondary outcomes or subgroup analyses when the primary outcome was statistically nonsignificant. Spin was more common in studies that had some component of industry funding.

EU requirements to provide results for authorised clinical trials – EC, EMA, MHA

Call for all sponsors to publish clinical trial results in EU database (joint letter by the European Commission, EMA and HMA)

 

The European Commission (EC), the European Medicines Agency (EMA) and the Heads of Medicines Agencies (HMA) have co-signed a letter reminding all sponsors of clinical trials conducted in the European Union of their obligation to make summaries of results of concluded trials publicly available in the EU Clinical Trials Database (EudraCT).

Transparency and public access to clinical trial results, whether positive or negative, are fundamental for the protection and promotion of public health. It assures trial subjects that their voluntary participation in clinical trials is useful and that the results have been collated and reported for the benefit of all. In addition, for those medicines which are placed on the market or used in further clinical trials, it allows patients and healthcare professionals, or any other citizen, to find out more information about medicines they might be taking or prescribing. Transparency also enhances scientific knowledge and helps to advance clinical research and support more efficient medicine development programmes.

It is the responsibility of sponsors to ensure that the protocol information and results of all clinical trials is submitted in EudraCT; this information is publicly available through the EU Clinical Trials Register (EU CTR). Since July 2014, sponsors are required to post results within one year after the end of a clinical trial (or six months for a paediatric trial). This information is also shared with the World Health Organization’s (WHO) International Clinical Trials Registry Platform (ICTRP) of which EU CTR is a primary registry.

As of April 2019, the EudraCT database included 57,687 clinical trials in total, out of which 27,093 were completed. Out of these completed trials, 18,432 should have had results posted; sponsors were in compliance with the publication requirements for 68.2% (12,577) of the trials, however results were still lacking for 31.8% of them (5,855).

The reporting compliance of non-commercial sponsors (e.g. academia) was much lower than for commercial sponsors (i.e. companies), with 23.6% of results posted for non-commercial sponsors vs 77.2% for commercial sponsors. Academic sponsors or smaller companies often lack awareness or incentives to post clinical results, therefore EU authorities are taking various steps to ensure sponsors are aware of their obligations and can act on them.

One of these initiatives is the “letter to stakeholders regarding the requirements to provide results for authorised clinical trials in EudraCT”, co-signed by Anne Bucher, Director General of the EC’s DG Health and Food Safety, Guido Rasi, Executive Director of EMA, and Thomas Senderovitz, Chair of the HMA Management Group. It will be disseminated to various stakeholder groups, with a goal in particular to reach academic sponsors. This should help to spread the word about the importance of making clinical trial results publicly available.

Amongst other initiatives conducted at EU level, EMA has since September 2018 been identifying trials with missing results on a monthly basis and sending reminders to the sponsors of those trials to ensure compliance with the transparency rules and their follow up on their results reporting obligations.

Reference.

Clinical Trials on Vaccines : the Revelations of a Doctor

Meehan MD on the poor science of the recent Danish MMR Autism Study

Published on 6 March 2019, by Jim Meehan, MD

 

The James Lind Library’s Introduction to Fair Tests of Treatments

JLL Book of Essays, About Fair Tests, May 2019

Introduction

At various times in our lives and to varying levels of intensity, we all use, provide or pay for health and social care. As we decide what to do, take, offer or buy, we need evidence that is reliable, robust and trustworthy about different options. Even before James Lind’s experiment comparing possible treatments for scurvy on HMS Salisbury people had recognised that getting this evidence requires strenuous efforts to reduce bias –but that achieving this is often not straightforward. This book of essays from the James Lind Library is our attempt to illustrate some of the challenges encountered and how to overcome them.

We will take you on a journey through the sometimes stormy waters of why treatments need to be tested, rather than being based on assumptions that “it must work” before the treatment has even been tried, or based on impressions after it has been used a few times, through to the need for fair tests comparing alternative treatment options. We will show why genuine uncertainties must be identified and addressed, and how research to find the most effective and appropriate treatments need to build on research to identify the most effective and appropriate methods for doing that research. We will navigate through the reasons why comparisons need to be fair at the outset, and then kept fair as the treatments being tested are given;outcomes are measured;and results are analysed, reported, and combined in systematic reviews of all the relevant, trustworthy evidence.

We have not cluttered the chapters with references to all the source material on which we have drawn. For that level of detail, please follow the links to the fuller essays on the James Lind Library website. Instead, where we know of reviews of methodology research which are relevant to a topic, we have listed these at the end of each chapter.

By the end of the book, we hope that you will recognise how, to bring benefits of research to patients and the public, systematic reviews of fair tests are needed to provide key elements of the knowledge needed to inform decisions about health and social care, while taking into account other important factors, such as values, preferences, needs, resources and priorities. We also hope that, as you finish the book, you will share the sense of enlightenment, education and enjoyment that we have gained from preparing it.

Finally, we dedicate this book to England’s National Institute for Health Research. Without the Institute’s 16-year-long support for the James Lind Initiative, the home of the James Lind Library during that time, neither the Library nor these essays would have been possible. And we also wish to acknowledge the role the Institute plays in recognising the vital contribution of research to the delivery of health and social care that is effective and efficient, and the Institute’s leadership in ensuring that the research itself is effective, efficient and reliable, with minimal waste.

Abstract

1.3 Why treatment comparisons must be fair

Untrustworthy treatment comparisons are those in which biases, or the play of chance, or both result in misleading estimates of the effects of treatments. Fair treatment comparisons avoid biases and reduce the effects of the play of chance.

It is not only failure to test theories about treatments in practice that has caused preventable tragedies. They have also occurred because the tests used to assess the effects of treatments have been unreliable and misleading. In the 1950s, theory and poorly controlled tests yielded unreliable evidence suggesting that diethylstilboestrol (DES) helped pregnant women who had previously had miscarriages and stillbirths. Although fair tests suggested that DES was useless, theory and unreliable evidence, together with aggressive marketing, led to DES being prescribed to millions of pregnant women over the next few decades. The consequences were disastrous for the women and their children, who experienced infertility and cancers as a result. The lesson is that a treatment that has not been reliably shown to be useful should not be promoted.

Problems resulting from inadequate tests of treatments continue to occur. Again, because of unreliable evidence and aggressive marketing, millions of women were persuaded to use hormone replacement therapy (HRT). It was claimed that, not only could it reduce unpleasant menopausal symptoms, but also the chances of having heart attacks and strokes. When these claims were assessed in fair tests, the results showed that in women over 60, far from reducing the risks of heart attacks and strokes, HRT increases the risks of these life-threatening conditions, as well as having other undesirable effects.These examples of the need for fair tests of treatments are a few of many that illustrate how treatments can do more harm than good. Improved general knowledge about fair tests of treatments is needed so that –laced with a healthy dose of scepticism –we can all assess claims about the effects of treatments more critically. That way, we will all become more able to judge which treatments are likely to do more good than harm.

DES DiEthylStilbestrol Resources

Meet the people who lived with defective devices

Implant Files, video published by ICIJ, 25 November 2018

ICIJ listened to hundreds of stories from patients with poorly-functioning medical devices. Here are just five of their testimonies.

Read more about ICIJ implant files investigation.

ICIJ “fights corruption with the world’s best cross-border watchdog journalism by over 160 investigative reporters in 60+ countries. The home of Offshore Leaks.”

Why join the Implant Files investigation ?

Implant Files, video published by ICIJ, 25 November 2018

Journalists explain why they decided to investigate medical device harm with ICIJ reporters.

Read more about ICIJ implant files investigation.

ICIJ “fights corruption with the world’s best cross-border watchdog journalism by over 160 investigative reporters in 60+ countries. The home of Offshore Leaks.”

A global investigation into medical device harm

Implant Files, video published by ICIJ, 24 November 2018

Patients around the world have become unwitting test subjects for new medical technology. Often following the trusted advice of their doctors, they have been injured, maimed and killed by poorly-tested implants.

Read more about the their implant files investigation.

ICIJ “fights corruption with the world’s best cross-border watchdog journalism by over 160 investigative reporters in 60+ countries. The home of Offshore Leaks”.