How we expect researchers to make all their data available
The movement to make data from clinical trials widely accessible has achieved enormous success, and it is now time for medical journals to play their part. From 1 July The BMJ will extend its requirements for data sharing to apply to all submitted clinical trials, not just those that test drugs or devices.
The BMJ’s Elizabeth Loder explains what this means for authors, and how we expect researchers to make their data available.
Ben Goldacre was speaking in the home of the BBC Proms on recent progress in promoting transparency in clinical trials
As part of its mission to promote access to the arts and sciences, the Royal Albert Hall hosted an historic event on Monday 21 April 2014, curating a spectacular and provocative programme of talks from world experts celebrating innovation, imagination, inspiration and their passion for a better future in health and medicine.
Ben Goldacre, former TED and TEDMED speaker, is a doctor, academic, campaigner and bad science writer whose work focuses on uses and misuses of science and statistics. He is the founder of the All Trials campaign and a keen advocate for clinical transparency.
Quanticate hosted a free symposium, which took place at the Royal College of Surgeons, September 2013
Dr Ben Goldacre, author of ‘Bad Pharma‘ presented a keynote speech at the Clinical Data Live! symposium. In this video hear Ben Goldacre present ‘ALLTrials: Transparency is moving forwards, industry can benefit from doing the right thing‘. This symposium was hosted by Quanticate who support Pharma and Biotech companies with statistical consultancy services.
Evidence for the Selective Reporting of Analyses and Discrepancies in Clinical Trials: A Systematic Review of Cohort Studies of Clinical Trials
Most publications about selective reporting in clinical trials have focussed on outcomes. However, selective reporting of analyses for a given outcome may also affect the validity of findings. If analyses are selected on the basis of the results, reporting bias may occur. The aims of this study were to review and summarise the evidence from empirical cohort studies that assessed discrepant or selective reporting of analyses in randomised controlled trials (RCTs). Methods and Findings
A systematic review was conducted and included cohort studies that assessed any aspect of the reporting of analyses of RCTs by comparing different trial documents, e.g., protocol compared to trial report, or different sections within a trial publication. The Cochrane Methodology Register, Medline (Ovid), PsycInfo (Ovid), and PubMed were searched on 5 February 2014. Two authors independently selected studies, performed data extraction, and assessed the methodological quality of the eligible studies. Twenty-two studies (containing 3,140 RCTs) published between 2000 and 2013 were included. Twenty-two studies reported on discrepancies between information given in different sources. Discrepancies were found in statistical analyses (eight studies), composite outcomes (one study), the handling of missing data (three studies), unadjusted versus adjusted analyses (three studies), handling of continuous data (three studies), and subgroup analyses (12 studies). Discrepancy rates varied, ranging from 7% (3/42) to 88% (7/8) in statistical analyses, 46% (36/79) to 82% (23/28) in adjusted versus unadjusted analyses, and 61% (11/18) to 100% (25/25) in subgroup analyses. This review is limited in that none of the included studies investigated the evidence for bias resulting from selective reporting of analyses. It was not possible to combine studies to provide overall summary estimates, and so the results of studies are discussed narratively.
Discrepancies in analyses between publications and other study documentation were common, but reasons for these discrepancies were not discussed in the trial reports. To ensure transparency, protocols and statistical analysis plans need to be published, and investigators should adhere to these or explain discrepancies.
Sources and Full Report
Evidence for the Selective Reporting of Analyses and Discrepancies in Clinical Trials: A Systematic Review of Cohort Studies of Clinical Trials, PLOS one Collections, DOI: 10.1371/journal.pmed.1001666, June 24, 2014.
The FPM shows overwhelming support for the principles of clinical trial transparency
Members of the UK Faculty of Pharmaceutical Medicine have expressed backing for greater transparency in clinical trials in their first survey on the topic. They showed clear support for registration of all clinical trials, earlier publication of the summary results, and increased access to trial data.
95% of survey respondents believe that all clinical trials should be registered
89% believe that increased publication of clinical trial results (including negative results) will ultimately lead to better medicines and better healthcare for patients
73% of respondents thought that clinical trials, irrespective of phase of development, should be published within 1– 2 years of completion (not linked to market authorisation or discontinuation of the project)
87% believe that, overall, increased scrutiny of clinical trial data will result in a stronger science base and enhance medical research
Only 10% believe that increased publication and dissemination of clinical trial results will harm the commercial environment in which companies operate
69% supported the requirement for retrospective release of clinical trials data , with the most commonly agreed time frame being data from ~5 years ago
Sources and More Information:
Clinical trials transparency and access to data,
Faculty of Pharmaceutical Medicine, update, 28.8.14.
Faculty of Pharmaceutical Medicine survey of members on transparency in clinical trials, FPM, Analysis Report, 28.8.14.
Survey of pharmaceutical physicians shows overwhelming support for the principles of clinical trial transparency, FPM press release.
Faculty of Pharmaceutical Medicine backs greater transparency in clinical trials, BMJ, 349/bmj.g5381, 01 September 2014.
All trials past and present should be registered, and the full methods and the results reported
It is time all clinical trial results are reported. Results from around half of clinical trials have never been published. Information on what was done and what was found in these trials could be lost forever to doctors and researchers, leading to bad treatment decisions, missed opportunities for good medicine, and trials being repeated. The contributions of the hundreds of thousands of patients who took part in those trials remain unused and unusable.
” I don’t think industry are concerned about patient confidentiality except in so far as they are concerned to avoid being sued for injuries in clinical trials. ”
There is a very informative and fascinating debate going on between Dr David Healy and Ben Goldacre regarding options, thoughts, strategies for better clinical trials access and transparency.
Dr. David Healy posted “fucked” here and I posted a summary here.
Dr Ben Goldacre responded here and here. I republished his reply here.
Dr. David Healy then clarified here – see below – I only added few related links (in the original text response) with the purpose to bring more clarity and/or references to the readers.
Again let me invite you to read “fucked” post 22 comments – about clinical trial data access and pharmaceutical industry transparency.
” The first point to make is this post isn’t about AllTrials.
AllTrials is a footnote. It’s about the dismay that many felt at EMA backsliding. It’s about how it was obvious that something like this was on the cards. Against this background uncritical endorsement of industry looked like a bad idea. There was a desperate need to stay awake. It looks like too many of us have been asleep. Ben offers an outline of the AllTrials strategy here. It’s helpful to have this. His accusation that these posts misrepresent campaigns, smear people, shout abuse, and hector from the sidelines looks like a description of posts by others elsewhere. With very few exceptions any comments to the various posts on this blog that in any way fail to support Ben or AllTrials have been deleted. The post repeated an alternate analysis – that the main thing industry wants to hide are adverse event data. In a post 18 months ago I outlined how to achieve this industry would in public deploy the issue of patient confidentiality as a main justification for hiding data. In this it seems to me they have been assisted by Iain Chalmers editorial with Patrick Vaillance and now by Ben. The historical evolution of the confidentiality issue is that the first informed consent forms said nothing about not showing your data to anyone else. Unnoticed industry have slipped in a “we will of course show your data to no-one clause”. At the EMA conference on data access in November 2012, I made two points. The second was that industry would assert the notion of their privacy rights – which they have done. The other was that no one signs to have their data sequestered. Afterwards, Iain Chalmers congratulated me on the point – I thought we were on the same page. Whether adverse event data is key or not, Peter Gotzsche through the European Ombudsman and Tom Jefferson and Peter Doshi through Tamiflu and RIAT seem to me to have done more in practical terms to move the issues forward than anyone else. It leaves me wondering why there is an endless call to celebrate Ben and not Peter or Tom. Some of us have been working the GSK system and can see what the pitfalls are. Even if not redacted, this is a system that will make it close to impossible to analyse CSRs properly. But if it’s not proclaimed by AllTrials first it seems like such insights are unwelcome. In several posts before the latest debacle I outlined how in my opinion there was a real chance that magnificent though he has been and clearly morally right, Peter Gotzsche’s efforts may do more harm than good. Even without taking GSK’s preposterous data access system into account, pushing for data adds to the undue premium being put on RCTs Twenty years ago the moral case for access was as strong and the risks consequent on failing were much less in that we were less hypnotized by RCTs than we are now. Far from responding shrilly, Peter Gotzsche recognized the risk and we have been collaborating ever more closely since. The issues are so complex we might all be making mistakes – the only people unwilling to concede this seem to be AllTrials. The push for data access remains morally compelling but there are other things that can be done that might be more effective. As the BBC program a week ago on Thalidomide, and previous posts here, make clear, industry fear a boycott more than anything else. It is the only thing they have ever responded to. At the moment the focus is on a bunch of bureaucrats in EMA, who aren’t there with a brief to protect us other than by regulating the wording of advertisements. The focus should be on doctors who treat patients. We could refuse to use drugs where there is no access to the data. It shouldn’t even take courage to do this. In my opinion, this is the call that’s needed now rather than a call to support more of what AllTrials have been doing. But who will lead such a call? Along with colleagues I put forward a softer version of a boycott – an AbbVie – which encouraged doctors and patients to use drugs but to report on the adverse events which would in fact make these chemicals better medicines. It would be difficult for government or anyone else to gainsay this win-win option in the way they might come out against the lose-lose of a boycott. There is a conflict of interest here. RxISK.org has a stake in this idea. It was set up before AllTrials to move ideas like this forward. I suspect those of us working on RxISK in the evenings and at weekends have been putting far more hours into the effort than the AllTrials team have. At the end of the day, I may well be wrong on this, but I personally think AllTrials have been naïve. I don’t think industry are concerned about patient confidentiality except in so far as they are concerned to avoid being sued for injuries in clinical trials. Recent decades have seen industry put Litigation Support Defences in place. As outlined a decade ago in Let Them Eat Prozac, putting a premium on clinical trials has been a key element in their litigation support strategy. Seen from this vantage point AllTrials offers Pharma a lot – all without the effort of having to conspire or fund a conspiracy. Playing straight into industry’s hands is a hazard for all of us. Good intentions aren’t enough to save us. I’d rest more comfortably if the key players in AllTrials had a track record in bringing adverse events to light or even a record of supporting those trying to do so – if they’d really antagonized industry good and proper. It’s not that partnership isn’t nice but perhaps after playing hard to get first. ”