The Myth of Scientific Objectivity

Dr. Terence Kealey during a CrossFit Health event on March 9, 2019

“I think there is a vast myth that scientists are somehow objective and honest.”

~Dr. Terence Kealey~

Kealey is a former vice-chancellor of the University of Buckingham, a professor of clinical biochemistry, a scholar affiliated with the Cato Institute, and author of the book Breakfast Is a Dangerous Meal. During his presentation, he discussed the myth of scientific objectivity, drawing examples widely from history as well as his personal experiences within many of the most reputable scientific institutions.

Video published on 12 Aug 2019. Reference. Full transcript.

Spin in Published Biomedical Literature : A Systematic Review

Medical research publications : how to modify (abstracts) perception of results

Quinn Grundy presents original research that explores the nature and prevalence of spin in the biomedical literature. Video published on 11 Oct 2017. Reference.

Objective
To explore the nature and prevalence of spin in the biomedical literature.

Design
In a systematic review and meta-analysis, we searched MEDLINE, PreMEDLINE, Embase, Scopus, and handsearched reference lists for all articles published between 1946 and 24 November 2016 that included the quantitative measurement of spin in the biomedical literature for at least 1 outcome. Two independent coders extracted data on the characteristics of articles and included studies, methods for assessing spin, and all spin-related results. The data were heterogeneous; results were grouped inductively into outcome-related categories. We had sufficient data to use meta-analysis to analyze the association of industry sponsorship of research with the presence of spin.

Results
We identified 4219 articles after removing duplicates and included 35 articles that investigated spin: clinical trials (23/35, 66%); observational studies (7/35, 20%); diagnostic accuracy studies (2/35, 6%); and systematic reviews and meta-analyses (4/35, 11%), with some articles including multiple study designs. The nature and manifestations of spin varied according to study design. We grouped results into the following categories: prevalence of spin, level of spin, factors associated with spin, and effects of spin on readers’ interpretations. The highest, but also greatest variability in the prevalence of spin was present in trials (median, 57% of main texts containing spin; range, 19%-100% across 16 articles). Source of funding was hypothesized to be a factor associated with spin; however, the meta-analysis found no significant association, possibly owing to the heterogeneity of the 7 included articles.

Conclusions
Spin appears to be common in the biomedical literature, though this varies by study design, with the highest rates found in clinical trials. Spin manifests in diverse ways, which challenged investigators attempting to systematically identify and document instances of spin. Widening the investigation of factors contributing to spin from characteristics of individual authors or studies to the cultures and structures of research that may incentivize or deincentivize spin, would be instructive in developing strategies to mitigate its occurrence. Further research is also needed to assess the impact of spin on readers’ decision making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research to ensure accurate interpretation and dissemination of research.

Identification of Spin in Clinical Studies Evaluating Biomarkers in Ovarian Cancer

Medical research publications : how to modify (abstracts) perception of results

Mona Ghannad presents a systematic review which documents and classifies spin or overinterpretation, as well as facilitators of spin, in recent clinical studies evaluating performance of biomarkers in ovarian cancer. Video published on 11 Oct 2017. Reference.

Objective
The objective of this systematic review was to document and classify spin or overinterpretation, as well as facilitators of spin, in recent clinical studies evaluating performance of biomarkers in ovarian cancer.

Design
We searched PubMed systematically for all studies published in 2015. Studies eligible for inclusion described 1 or more trial designs for identification and/or validation of prognostic, predictive, or diagnostic biomarkers in ovarian cancer. Reviews, animal studies, and cell line studies were excluded. All studies were screened by 2 reviewers. To document and characterize spin, we collected information on the quality of evidence supporting the study conclusions, linking the performance of the marker to outcomes claimed.

Results
In total, 1026 potentially eligible articles were retrieved by our search strategy, and 345 studies met all eligibility criteria and were included. The first 200 studies, when ranked according to publication date, will be included in our final analysis. Data extraction was done by one researcher and validated by a second. Specific information extracted and analyzed on study and journal characteristics, key information on the relevant evidence in methods, and reporting of conclusions claimed for the first 50 studies is provided here. Actual forms of spin and facilitators of spin were identified in studies trying to establish the performance of the discovered biomarker.

Actual forms of spin identified as shown (Table) were:

  1. other purposes of biomarker claimed not investigated (18 of 50 studies [36%]);
  2. incorrect presentation of results (15 of 50 studies [30%]);
  3. mismatch between the biomarker’s intended clinical application and population recruited (11 of 50 studies [22%]);
  4. mismatch between intended aim and conclusion (7 of 50 studies [14%]);
  5. and mismatch between abstract conclusion and results presented in the main text (6 of 50 studies [12%]).

Frequently observed facilitators of spin were:

  1. not clearly prespecifying a formal test of hypothesis (50 of 50 studies [100%]);
  2. not stating sample size calculations (50 of 50 studies [100%]);
  3. not prespecifying a positivity threshold of continuous biomarker (17 of 43 studies [40%]);
  4. not reporting imprecision or statistical test for data shown (ie, confidence intervals, P values) (12 of 50 studies [24%]);
  5. and selective reporting of significant findings between results for primary outcome reported in abstract and results reported in main text (9 of 50 studies [18%]).

Conclusions
Spin was frequently documented in abstracts, results, and conclusions of clinical studies evaluating performance of biomarkers in ovarian cancer. Inflated and selective reporting of biomarker performance may account for a considerable amount of waste in the biomarker discovery process. Strategies to curb exaggerated reporting are needed to improve the quality and credibility of published biomarker studies.

Systematic reviews with financial conflicts of interest linked to favourable conclusions and lower methodological quality

Financial conflicts of interest in systematic reviews: associations with results, conclusions, and methodological quality

Abstract

Background
Financial conflicts of interest in systematic reviews (e.g. funding by drug or device companies or authors’ collaboration with such companies) may impact on how the reviews are conducted and reported.

Objectives
To investigate the degree to which financial conflicts of interest related to drug and device companies are associated with results, conclusions, and methodological quality of systematic reviews.

Search methods
We searched PubMed, Embase, and the Cochrane Methodology Register for studies published up to November 2016. We also read reference lists of included studies, searched grey literature sources, and Web of Science for studies citing the included studies.

Selection criteria
Eligible studies were studies that compared systematic reviews with and without financial conflicts of interest in order to investigate differences in results (estimated treatment effect and frequency of statistically favourable results), frequency of favourable conclusions, or measures of methodological quality of the review (e.g. as evaluated on the Oxman and Guyatt index).

Data collection and analysis
Two review authors independently determined the eligibility of studies, extracted data, and assessed risk of bias. We synthesised the results of each study relevant to each of our outcomes. For meta‐analyses, we used Mantel‐Haenszel random‐effects models to estimate risk ratios (RR) with 95% confidence intervals (CIs), with RR > 1 indicating that systematic reviews with financial conflicts of interest more frequently had statistically favourable results or favourable conclusions, and had lower methodological quality. When a quantitative synthesis was considered not meaningful, results from individual studies were summarised qualitatively.

Main results
Ten studies with a total of 995 systematic reviews of drug studies and 15 systematic reviews of device studies were included. We assessed two studies as low risk of bias and eight as high risk, primarily because of risk of confounding. The estimated treatment effect was not statistically significantly different for systematic reviews with and without financial conflicts of interest (Z‐score: 0.46, P value: 0.64; based on one study of 14 systematic reviews which had a matched design, comparing otherwise similar systematic reviews). We found no statistically significant difference in frequency of statistically favourable results for systematic reviews with and without financial conflicts of interest (RR: 0.84, 95% CI: 0.62 to 1.14; based on one study of 124 systematic reviews). An analysis adjusting for confounding due to methodological quality (i.e. score on the Oxman and Guyatt index) provided a similar result. Systematic reviews with financial conflicts of interest more often had favourable conclusions compared with systematic reviews without (RR: 1.98, 95% CI: 1.26 to 3.11; based on seven studies of 411 systematic reviews). Similar results were found in two studies with a matched design, which therefore had a reduced risk of confounding. Systematic reviews with financial conflicts of interest tended to have lower methodological quality compared with systematic reviews without financial conflicts of interest (RR for 11 dimensions of methodological quality spanned from 1.00 to 1.83). Similar results were found in analyses based on two studies with matched designs.

Authors’ conclusions
Systematic reviews with financial conflicts of interest more often have favourable conclusions and tend to have lower methodological quality than systematic reviews without financial conflicts of interest. However, it is uncertain whether financial conflicts of interest are associated with the results of systematic reviews. We suggest that patients, clinicians, developers of clinical guidelines, and planners of further research could primarily use systematic reviews without financial conflicts of interest. If only systematic reviews with financial conflicts of interest are available, we suggest that users read the review conclusions with skepticism, critically appraise the methods applied, and interpret the review results with caution.

Medical research publications : how to modify (abstracts) perception of negative (or non-significant) results into positive ones

Evaluation of Spin in the Abstracts of Emergency Medicine Randomized Controlled Trials

Spin is common (>40%) in emergency medicine randomized controlled trials…

May 2019 Study objective

We aim to investigate spin in emergency medicine abstracts, using a sample of randomized controlled trials from high-impact-factor journals with statistically nonsignificant primary endpoints.

Methods

This study investigated spin in abstracts of emergency medicine randomized controlled trials from emergency medicine literature, with studies from 2013 to 2017 from the top 5 emergency medicine journals and general medical journals. Investigators screened records for inclusion and extracted data for spin. We considered evidence of spin if trial authors focused on statistically significant results, interpreted statistically nonsignificant results as equivalent or noninferior, used favorable rhetoric in the interpretation of nonsignificant results, or claimed benefit of an intervention despite statistically nonsignificant results.

Results

Of 772 abstracts screened, 114 randomized controlled trials reported statistically nonsignificant primary endpoints. Spin was found in 50 of 114 abstracts (44.3%). Industry-funded trials were more likely to have evidence of spin in the abstract (unadjusted odds ratio 3.4; 95% confidence interval 1.1 to 11.9). In the abstracts’ results, evidence of spin was most often due to authors’ emphasizing a statistically significant subgroup analysis (n=9). In the abstracts’ conclusions, spin was most often due to authors’ claiming they accomplished an objective that was not a prespecified endpoint (n=14).

Conclusion

Spin was prevalent in the selected randomized controlled trial, emergency medicine abstracts. Authors most commonly incorporated spin into their reports by focusing on statistically significant results for secondary outcomes or subgroup analyses when the primary outcome was statistically nonsignificant. Spin was more common in studies that had some component of industry funding.

Pharma sales rep paid physicians to participate in hundreds of sham “speaker programs” in order to prescribe med drugs

Subsys : Drug Company Sales Rep Sentenced for Role in Kickback Scheme Related to Fentanyl Spray Prescriptions

John H. Durham, United States Attorney for the District of Connecticut, announced that NATALIE LEVINE, 35, of Scottsdale, Arizona, was sentenced today by U.S. District Judge Janet Bond Arterton in New Haven to five years of probation for engaging in a kickback scheme related to fentanyl spray prescriptions. Judge Arterton also ordered Levine to spend the first six months of probation in home confinement, and to perform 150 hours of community service.

According to court documents and statements made in court, from approximately March 2013 to October 2014, Levine was employed by Insys Therapeutics, an Arizona-based pharmaceutical company that manufactured and sold Subsys, a fentanyl-based sublingual spray that was approved by the Food and Drug Administration solely for the management of breakthrough pain in cancer patients. Levine was a sales representative for the company and was responsible for covering the territories that included Connecticut, New Hampshire and Rhode Island.

Levine induced certain medical practitioners, including an advanced practice registered nurse (APRN) in Connecticut, a physician’s assistant (PA) in New Hampshire, and a physician in Rhode Island, to prescribe Subsys by paying them to participate in hundreds of sham “Speaker Programs.” The Speaker Programs, which were typically held at high-end restaurants, were ostensibly designed to gather licensed healthcare professionals who had the capacity to prescribe Subsys and educate them about the drug. In truth, the events were usually just a gathering of friends and co-workers, most of whom did not have the ability to prescribe Subsys, and no educational component took place. “Speakers” were paid a fee that ranged from $1,000 to several thousand dollars for attending these dinners. At times, the sign-in sheets for the Speaker Programs were forged so as to make it appear that the programs had an appropriate audience of healthcare professionals.

The medical practitioners were paid thousands of dollars in illegal kickbacks in order to prescribe Subsys, and induce others to prescribe Subsys, over similar medications. Medicare Part D plans authorized payment for hundreds of Subsys prescriptions written by the three medical practitioners, resulting in a loss of approximately $4.5 million.

Levine’s restitution figure will be determined after additional court proceedings.

On July 11, 2017, Levine pleaded guilty to one count of conspiracy to violate the anti-kickback law.

Several other individuals affiliated with Insys Therapeutics, and medical practitioners involved in this kickback scheme, have been charged and convicted in the District of Connecticut and in other Districts across the United States. In sentencing Levine, Judge Arterton credited Levine’s significant cooperation and assistance to the government’s prosecution of defendants in Connecticut, Massachusetts, New Hampshire and Rhode Island.

On January 3, 2019, Levine’s husband, Michael Babich, who was the CEO and President of Insys Therapeutics, pleaded guilty in the District of Massachusetts to conspiracy and fraud charges stemming from the scheme. He awaits sentencing.

On May 2, 2019, a federal jury in Boston found John N. Kapoor, the founder and former Executive Chairman of Insys Therapeutics, and four other former Insys executives guilty of racketeering conspiracy.

Earlier this month, Insys Therapeutics agreed to pay a total of $225 million to resolve criminal and civil investigations of the company.

The investigation in the District of Connecticut is being conducted by the U.S. Department of Health and Human Services Office of the Inspector General and the Federal Bureau of Investigation, with the assistance of the Drug Enforcement Administration’s Tactical Diversion Squad. The case is being prosecuted by Assistant U.S. Attorneys Douglas P. Morabito, Sarah P. Karwan and Richard M. Molot.

U.S. Attorney Durham encouraged individuals who suspect health care fraud to report it by calling the Health Care Fraud Task Force (203) 785-9270 or 1-800-HHS-TIPS.

Department of Justice
U.S. Attorney’s Office, District of Connecticut, News And Press Releases, Monday June 24, 2019

Carey Gillam on Monsanto Lawsuit

The legal picture and the bigger picture, MARCH 29, 2019

How much power will we grant profit-driven corporations to determine not just what we’re exposed to, but what we’re permitted to know about it ?

Carey Gillam, longtime journalist, is currently research director at the group US Right to Know, and author of the book Whitewash: The Story of a Weedkiller, Cancer and the Corruption of Science.

Reference.

Astroturfing : les fichiers secrets de Monsanto

Enquête de Laura Aguirre de Carcer, Tristan Waleckx et les équipes d’”Envoyé spécial” diffusée le 16 mai 2019

Nouvelles révélations sur les documents secrets de Monsanto. Après le fichage illégal de journalistes, de chercheurs, de politiques, l’équipe d'”Envoyé spécial” a découvert une autre stratégie : l’astroturfing.

Des agences de communication travaillant pour Monsanto ont créé un “faux mouvement spontané” de défense du glyphosate. Des hôtesses ont ainsi recruté des agriculteurs sans jamais leur dire qu’elles travaillaient pour la firme. L’objectif était de créer un groupe d’influence venant du terrain, en dissimulant cette stratégie aux premiers concernés : les agriculteurs. Référence.

 

With Pharma funding the MHRA, can the EWG Report be Fully Independent ?

Westminster Hall debate, 23 April 2019

” Yes we had a (Primodos) review, but I’m not sure we can call that ‘independant’ “

Yasmin Qureshi MP calls into question the independence of the MHRA in its central role on the expert working group (EWG), for their report on the use of hormone pregnancy tests (HPTs) and adverse effects relating to pregnancy including possible birth defects.

Westminster Hall debate, 23 April 2019. Video reference.

More information

Dealing with strategies used by industry to influence scientific evidence

Commercial interests, transparency, and independence: a call for submissions

Help the move towards independence from commercial interests

A decade ago the US Institute of Medicine (IOM) issued a landmark report on conflicts of interest in research, medical education, and practice.1 Highlighting benefits of collaborations between physicians, researchers, and companies to develop new products that can improve health, the report also raised substantial concerns that extensive financial ties could unduly influence professional judgments. It concluded these financial conflicts of interest could jeopardise the integrity of science, the objectivity of education, the quality of care, and public trust in medicine. The report recommended more research on conflicts of interest, improvements in transparency, and greater independence from industry.

Today we announce plans for a stream of BMJ content to revisit these concerns and ask you to join us. A key aim is to identify and respond to commercial influences on health and healthcare, to understand under what circumstances involvement with industry is truly necessary. Where it is not necessary, we want to forge a new independence from those who make and sell products, to strengthen trust in how evidence is produced and disseminated, and to drive more rational and safer use of drugs, devices, diagnoses, and data in the public interest.

Key Points

  • Problematic relationships
  • The BMJ’s response so far
  • Call for submissions

Read