It is perhaps predictable that in an era characterized by a growing expectation that health decisions should be evidence-based, fake news is, as the Merriam Webster dictionary says, “having a bit of a moment”.
At a time when clinicians, policy-makers and consumers are increasingly looking for evidence to support their decisions about health, this “false, often sensational, information” is all around us and it is becoming increasingly unclear what information can be trusted.
Cochrane Reviews have always aimed to be the gold standard in systematic reviews of health research; to provide trusted evidence to guide health decisions. In the growing murk of misleading marketing materials and other subtly or blatantly biased (or just-plain-wrong) health information, now more than ever it is vital that Cochrane Reviews remain trusted, and trustworthy, evidence.
Now more than ever it is vital that Cochrane Reviews remain trusted, and trustworthy, evidence
To help ensure Cochrane Reviews remain trustworthy, and can be produced efficiently and sustainably, we conducted a series of interviews and a survey of people involved in producing Cochrane Reviews. We hoped to understand how the process of producing Cochrane Reviews currently works, and what could be done to improve it. The results of this research are published today in Systematic Reviews.
The findings emphasize some things that have been known anecdotally for some time, and especially that creating reliable, timely systematic reviews is incredibly hard-work.
We also identified challenges with the increasing complexity of review methods; with keeping authors on board and on track; and with how long it takes to complete the review process.
There were strong messages about the roles of authors and Review Groups, the main players in review production. There are clear needs to improve clarity of roles and expectations; to ensure continuity and consistency of input; and to support active management of the review process.
It was suggested that Cochrane might consider better ways of building the capacity of and sharing information between authors and Review Groups, and changing the way we work by, for example, breaking reviews into smaller “chunks”.
While the results of this research are, intentionally, primarily relevant to Cochrane Reviews, similar challenges are faced by all systematic reviewers, and the work of developing the solutions will also be shared.
It was suggested that Cochrane might consider better ways of building the capacity of and sharing information between authors and Review Groups
It’s exciting to see people, within Cochrane and beyond, already trying new approaches, and looking for clever solutions to some of these challenges. Production innovations like Living Systematic Reviews, Rapid Reviews and Cochrane Targeted Updates are exploring how systematic reviews can be as useful and timely as possible, while maintaining rigor and reliability.
There is also an important role for new technologies to play in enabling improvements in review production. Cochrane Evidence Pipeline and Cochrane Crowd are already having a substantial impact on reducing the effort required to search for and screen studies for Cochrane Reviews, and Linked Data work is changing the way we think about the data included in our reviews.
There is plenty of work still to do to improve the production of Cochrane Reviews, and that work will be ongoing. The Shorenstein Center at Harvard University says that a key step in combating fake news is “Making the Truth “Louder”” by acting to “strengthen trustworthy sources of information”. We certainly hope that this research, and Cochrane’s response to it, will help to do that.