Recruiting participants to evaluate reproducibility

There has been a lot of discussion and attention on reproducibility in science. To try and overcome this issue, particularly for cancer research, the Reproducibility Project: Cancer Biology was announced and those working on the project talk more about it in this blog.

Basic and preclinical research generates exciting, new ideas that drive the scientific discovery process and evaluates potential interventions in cells and animals. It forms the foundation of scientific claims that future research is built on and eventually leads to the formation of new therapies and clinical studies.

However, recently there has been a lot of discussion and attention about reproducibility in science. In biomedical research there are increasing concerns about the inability to reproduce and replicate major findings.

What is this reproducibility project?

While there has been growing evidence of irreproducibility in biomedical research the evidence is incomplete. Just as any scientific claim needs to be transparent in process and evidence to support that claim, the same applies to whether something is reproducible, or not.

In 2013 the Reproducibility Project: Cancer Biology (RP:CB), a collaboration between the Center for Open Science and Science Exchange, was announced to independently replicate selected results from a substantial number of high-profile papers in the field of cancer biology.

The project uses a transparent and open methodology to examine reproducibility and will result in a dataset that will provide evidence about reproducibility in cancer biology.

The project uses a transparent and open methodology to examine reproducibility and will result in a dataset that will provide evidence about reproducibility in cancer biology, and an opportunity to identify factors that influence reproducibility more generally. This follows a protocol similar to the Reproducibility Project: Psychology.

For each Replication Study, a Registered Report detailing the proposed experimental designs, protocols, samples sizes, and analysis plans for the replications is peer reviewed and published prior to data collection through a partnership with the open-access journal eLife.

The results of these experiments will be published as a Replication Study by eLife and will contain all data, analyses, detailed descriptions of all procedures, and any deviations that may have occurred from the Registered Report during the experimental phase.

What is a successful replication?

This seemingly simple question is not sufficiently described by any single indicator. Because success in replication is complex, the RP:CB aims to assess replication using multiple approaches.

We are seeking to recruit scientists in biomedical research to review the results of the replications and give their subjective opinion of whether the original result was successfully replicated.

One of these is subjective assessment. We are seeking to recruit scientists in biomedical research to review the results of the replications and give their subjective opinion of whether the original result was successfully replicated.

Currently we are collecting a list of interested scientists. A formal survey will be available before the replication results are made public. We will reach out to scientists on our list at that time to take the survey.

The project is still ongoing, so it might be a few weeks, months, or sometime next year when a survey becomes available. If interested, view the active replications and go to https://cos.io/rpcb/ to register for the subjective assessment.

View the latest posts on the Research in progress blog homepage

Comments