Studies need to be reported in a detailed and transparent fashion. The ultimate test of this is whether enough information has been provided to allow potential replication or, at the very least, to permit a reader to make a robust assessment of the study findings.
Reporting checklists, such as PRISMA and CONSORT, together with initiatives such as the EQUATOR network are very helpful in this regard. However, despite these checklists the standard of reporting continues to be less than ideal.
Papers are continually submitted to journals with woefully inadequate methodological detail, abstracts are poorly reported, and there is often little justification for a chosen methodological approach. In the current era of electronic publishing, where there is little limit to the amount of supplementary material that can be provided, there is no excuse.
Improving the quality of studies
The MAPS reporting statement is a checklist aimed at improving the quality of studies that report estimates of utility (i.e. the Q in the QALY), through a process called mapping. This then informs an assessment of the number of quality adjusted life years (QALYs) associated with healthcare interventions in a cost-utility analysis.
Utility estimates can be derived in a number of different ways. Most reimbursement agencies prefer utility estimates to be derived directly through the use of a preference-based quality of life measure, such as the EQ-5D.
However, when such data are not available then indirect approaches, such as mapping, are an alternative. Indeed, Tosh et al. reported that over 25% of submissions to NICE had used mapping.
Mapping is a process where an algorithm is used to link data from a non-preference based quality of life measure, such as the Health Assessment Questionnaire, to a preference based quality of life measure, such as the EQ-5D.
Mapping is a complex process, involves a number of assumptions and methodological choices, and introduces further layers of uncertainty into cost-utility analyses.
The lack of internal validity
Methods commonly used in health economics and outcomes research lack the internal validity of a standard phase III randomised study design. As such there are more opportunities to introduce bias, as many methodological choices are left to the analyst’s discretion, leaving plenty of opportunities to manipulate findings.
In addition, analyses are often conducted with a very specific population in mind. Mapping is no different. Therefore, I would argue that it is even more important to fully report these study types.
This will not only allow replication and critical appraisal, but will also allow decision makers to assess the applicability of the findings to their particular population and to avoid the potential opportunity costs of making wrong decisions.
A welcome addition to the literature
Like CHEERs, the MAPS reporting statement, is a welcome addition to the reporting guideline literature. The statement was created using well-known methods including a Delphi Panel and a face to face meeting.
It includes a comprehensive and clearly worded checklist. There is also an explanation and elaboration document, published in PharmacoEconomics, which provides examples from the literature on how to meet each of the 23 reporting items.
However, reporting checklists are somewhat futile if authors, reviewers and journal editors do not put them into practice. Checklists need to be completed correctly so that when editors and reviewers go to the reported page number the item is present. Editors and reviewers need to use the checklists in their reviews of papers and hold authors to account where reporting is insufficient.
It will be very interesting to see whether the quality of reporting has improved when the authors revisit the need for a checklist in 5 years’ time. Given the MAPS reporting statement is published open access and in a number of well-known journals there really is no excuse for an improvement not to be observed.
Comments