“We spend a lot of time oiling and polishing this machine, but we have no idea if it’s working”. This quote succinctly summarizes how a lot of editors feel about the peer review process. It comes from Drummond Rennie, Deputy Editor of JAMA and an adjunct Professor of Medicine in the Institute for Health Policy Studies, University of California in San Francisco, USA. To find out more about the internal workings of the machine, Drummond initiated research into peer review and founded the International Congress on Peer Review and Biomedical Publication in 1986. Every four years, this event brings together Researchers, Editors and Publishers to hear about, present and discuss all aspects of scientific publishing.

Areas covered by the 12 plenary sessions at this year’s meeting in Chicago included authorship and citations, peer review, publication misconduct, bias and transparency in reporting, data sharing and post-publication access and dissemination. For the first time, BioMed Central was able to contribute its own research with two posters. One of us (Maria Kowalczuk) presented a study comparing the quality of peer review reports from an open and a closed peer review journal. This was done in collaboration with our fellow Biology and Medical Editors and the help of expert statistician Frank Dudbridge at London School of Hygiene and Tropical Medicine. Jigisha Patel presented a poster on work done in collaboration with Elizabeth Wager, a Publications Consultant at Sideview, and Iain Hrynaszkiewicz (a former colleague at BioMed Central and now at F1000) and the editorial staff of BMC Medicine, on the use and feasibility of incorporating the Anti-Ghostwriting Checklist into the manuscript process of a journal.

Joel Gagnier (University of Michigan, USA) and colleagues reported on the newly developed consensus based guidelines for the reporting of case reports which were subsequently published in the Journal of Medical Case Reports. While our medical BMC-Series journals by virtue of their use of open peer review where all versions of the manuscript and peer review reports are published alongside the paper, were the subject of a study conducted by Sally Hopewell (Oxford University, UK) and her colleagues, who looked at the extent to which manuscripts are revised in response to reviewers’ reports.

Our prize for novelty must go to Isabelle Boutron (INSERM, Paris, France) and colleagues for their presentation on the “Impact of Spin in the Abstract on the Interpretation of Randomized Controlled Trials in the Field of Cancer: A Randomised Controlled Trial”, which worryingly showed that the way trials were reported can convince readers that the beneficial effects of the experimental intervention were greater than shown by the results.

Among the most worrying issues raised at the meeting were the under-reporting of conflicts of interest, selective reporting in systematic reviews, discrepancies between trial results reported in ClinicalTrials.gov and subsequent published articles, poor methodological reporting, and poor uptake and use of the CONSORT guidelines.

Overall it was an immensely enjoyable meeting! However, we and many other editors felt slightly unnerved that despite the existence of good practice and reporting guidelines and checklists there is little evidence of uptake or adherence by authors, and that poor practices are still very prevalent across the scientific publishing landscape. Education and increasing awareness of these issues may go some way to addressing this problem, but perhaps it is time for publishers and journal editors to contribute by examining and innovating their own practices too. The Peer Review Congress was an excellent opportunity to exchange ideas. We look forward to attending the next Congress in 2017.

Maria Kowalczuk (Deputy Biology Editor) and Jigisha Patel (Medical Editor)