Medical research is important and benefits patients and public health.
Confirmation of scientific findings has been the hallmark of science; however, large scale clinical trials are not or rarely repeated. Therefore every single research result must be disseminated in consistency with the a priori plans.
A study may be ever so perfectly performed, but if the communication of the results is twisted to support post hoc hypotheses, it will undermine the very foundation of the research concept.
A priori plans versus post hoc hypotheses
In our study, published in Trials, we compared clinical trial study protocols and the corresponding results publication and found a flappergasting inconsistency in 61% of the cases examined.
We compared clinical trial study protocols and the corresponding results publication and found a flappergasting inconsistency in 61% of the cases examined.
The reasons we initiated our study were three-fold; a growing awareness of the importance of consistency between study protocols and publications, the devastating results of deviations therefrom that have been pointed out by trialists and methodologists, and indications that there are often inconsistencies between initial plans and the published results.
These concerns are supported by studies, which mainly focus on trials performed by the pharmaceutical industry, where a financial interest in the research results could be a potential problem.
These concerns called for a higher degree of transparency in medical research and led to the introduction of the International Committee of Medial Journal Editors (ICMJE) recommendation that journal editors should require all clinical trials are registered in a public registry before enrollment of patients in a study, i.e. with clearly defined a priori hypotheses.
We wanted to investigate if such problems also existed in academic trials where the economic interests are much less.
For this purpose, 95 pairs of protocols and corresponding publications were identified from a pool of 282 applications of academic drug trials submitted to the Danish Health and Medicines Authority in selected years between 1999 and 2003. This period ensured sufficient time for the publications to appear.
In order to assess the consistency, we identified and assessed five important characteristics for each protocol and their corresponding publications. These were; study type (was it an exploratory or confirmatory trial?), primary objective, primary end point – and for confirmatory studies only – hypothesis to be tested and sample size calculation.
Consistency on all five characteristics was found in only 39% of the studies, leading to the conclusion that one or more important discrepancies were present in 61% of the academic studies.
Besides raising awareness of a worrisome problem regarding the quality of the publications, our findings also indicate that the problem of not adhering to a priori hypotheses and plans is not confined to the pharmaceutical industry.
Besides raising awareness of a worrisome problem regarding the quality of the publications, our findings also indicate that the problem of not adhering to a priori hypotheses and plans is not confined to the pharmaceutical industry. It is in fact a more general problem, and therefore not solely attributed to economic interests.
The problem also seems to be independent of medical specialties being investigated.
The underlying reasons would probably be very difficult to identify, but could include pressure for publication combined with insufficient training, lacking focus on full transparency and limitations in open access to trial details.
Our findings call for increased transparency in all clinical trials, to be promoted by all stakeholders, including researchers, industry, journal editors and regulatory bodies. We also believe that studies like ours should ideally be repeated on a regular basis in order to monitor the current status of the quality of publications.
Comments