Dr. Terence Kealey during a CrossFit Health event on March 9, 2019
“I think there is a vast myth that scientists are somehow objective and honest.”
~Dr. Terence Kealey~
Kealey is a former vice-chancellor of the University of Buckingham, a professor of clinical biochemistry, a scholar affiliated with the Cato Institute, and author of the book Breakfast Is a Dangerous Meal. During his presentation, he discussed the myth of scientific objectivity, drawing examples widely from history as well as his personal experiences within many of the most reputable scientific institutions.
Medical research publications : how to modify (abstracts) perception of results
Quinn Grundy presents original research that explores the nature and prevalence of spin in the biomedical literature. Video published on 11 Oct 2017. Reference.
Objective To explore the nature and prevalence of spin in the biomedical literature.
Design In a systematic review and meta-analysis, we searched MEDLINE, PreMEDLINE, Embase, Scopus, and handsearched reference lists for all articles published between 1946 and 24 November 2016 that included the quantitative measurement of spin in the biomedical literature for at least 1 outcome. Two independent coders extracted data on the characteristics of articles and included studies, methods for assessing spin, and all spin-related results. The data were heterogeneous; results were grouped inductively into outcome-related categories. We had sufficient data to use meta-analysis to analyze the association of industry sponsorship of research with the presence of spin.
Results We identified 4219 articles after removing duplicates and included 35 articles that investigated spin: clinical trials (23/35, 66%); observational studies (7/35, 20%); diagnostic accuracy studies (2/35, 6%); and systematic reviews and meta-analyses (4/35, 11%), with some articles including multiple study designs. The nature and manifestations of spin varied according to study design. We grouped results into the following categories: prevalence of spin, level of spin, factors associated with spin, and effects of spin on readers’ interpretations. The highest, but also greatest variability in the prevalence of spin was present in trials (median, 57% of main texts containing spin; range, 19%-100% across 16 articles). Source of funding was hypothesized to be a factor associated with spin; however, the meta-analysis found no significant association, possibly owing to the heterogeneity of the 7 included articles.
Conclusions Spin appears to be common in the biomedical literature, though this varies by study design, with the highest rates found in clinical trials. Spin manifests in diverse ways, which challenged investigators attempting to systematically identify and document instances of spin. Widening the investigation of factors contributing to spin from characteristics of individual authors or studies to the cultures and structures of research that may incentivize or deincentivize spin, would be instructive in developing strategies to mitigate its occurrence. Further research is also needed to assess the impact of spin on readers’ decision making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research to ensure accurate interpretation and dissemination of research.
Medical research publications : how to modify (abstracts) perception of results
Mona Ghannad presents a systematic review which documents and classifies spin or overinterpretation, as well as facilitators of spin, in recent clinical studies evaluating performance of biomarkers in ovarian cancer. Video published on 11 Oct 2017. Reference.
Objective The objective of this systematic review was to document and classify spin or overinterpretation, as well as facilitators of spin, in recent clinical studies evaluating performance of biomarkers in ovarian cancer.
Design We searched PubMed systematically for all studies published in 2015. Studies eligible for inclusion described 1 or more trial designs for identification and/or validation of prognostic, predictive, or diagnostic biomarkers in ovarian cancer. Reviews, animal studies, and cell line studies were excluded. All studies were screened by 2 reviewers. To document and characterize spin, we collected information on the quality of evidence supporting the study conclusions, linking the performance of the marker to outcomes claimed.
Results In total, 1026 potentially eligible articles were retrieved by our search strategy, and 345 studies met all eligibility criteria and were included. The first 200 studies, when ranked according to publication date, will be included in our final analysis. Data extraction was done by one researcher and validated by a second. Specific information extracted and analyzed on study and journal characteristics, key information on the relevant evidence in methods, and reporting of conclusions claimed for the first 50 studies is provided here. Actual forms of spin and facilitators of spin were identified in studies trying to establish the performance of the discovered biomarker.
Actual forms of spin identified as shown (Table) were:
other purposes of biomarker claimed not investigated (18 of 50 studies [36%]);
incorrect presentation of results (15 of 50 studies [30%]);
mismatch between the biomarker’s intended clinical application and population recruited (11 of 50 studies [22%]);
mismatch between intended aim and conclusion (7 of 50 studies [14%]);
and mismatch between abstract conclusion and results presented in the main text (6 of 50 studies [12%]).
Frequently observed facilitators of spin were:
not clearly prespecifying a formal test of hypothesis (50 of 50 studies [100%]);
not stating sample size calculations (50 of 50 studies [100%]);
not prespecifying a positivity threshold of continuous biomarker (17 of 43 studies [40%]);
not reporting imprecision or statistical test for data shown (ie, confidence intervals, P values) (12 of 50 studies [24%]);
and selective reporting of significant findings between results for primary outcome reported in abstract and results reported in main text (9 of 50 studies [18%]).
Conclusions Spin was frequently documented in abstracts, results, and conclusions of clinical studies evaluating performance of biomarkers in ovarian cancer. Inflated and selective reporting of biomarker performance may account for a considerable amount of waste in the biomarker discovery process. Strategies to curb exaggerated reporting are needed to improve the quality and credibility of published biomarker studies.
Evaluation of Spin in the Abstracts of Emergency Medicine Randomized Controlled Trials
Spin is common (>40%) in emergency medicine randomized controlled trials…
May 2019 Study objective
We aim to investigate spin in emergency medicine abstracts, using a sample of randomized controlled trials from high-impact-factor journals with statistically nonsignificant primary endpoints.
Methods
This study investigated spin in abstracts of emergency medicine randomized controlled trials from emergency medicine literature, with studies from 2013 to 2017 from the top 5 emergency medicine journals and general medical journals. Investigators screened records for inclusion and extracted data for spin. We considered evidence of spin if trial authors focused on statistically significant results, interpreted statistically nonsignificant results as equivalent or noninferior, used favorable rhetoric in the interpretation of nonsignificant results, or claimed benefit of an intervention despite statistically nonsignificant results.
Results
Of 772 abstracts screened, 114 randomized controlled trials reported statistically nonsignificant primary endpoints. Spin was found in 50 of 114 abstracts (44.3%). Industry-funded trials were more likely to have evidence of spin in the abstract (unadjusted odds ratio 3.4; 95% confidence interval 1.1 to 11.9). In the abstracts’ results, evidence of spin was most often due to authors’ emphasizing a statistically significant subgroup analysis (n=9). In the abstracts’ conclusions, spin was most often due to authors’ claiming they accomplished an objective that was not a prespecified endpoint (n=14).
Conclusion
Spin was prevalent in the selected randomized controlled trial, emergency medicine abstracts. Authors most commonly incorporated spin into their reports by focusing on statistically significant results for secondary outcomes or subgroup analyses when the primary outcome was statistically nonsignificant. Spin was more common in studies that had some component of industry funding.
