Prospective publication of protocols and results of pilot and feasibility works

The next step to think about is to prospectively publish your protocol and publish your results. To talk us through this point, Lehana Thabane (Associate Chair of the Department of Clinical Epidemiology and Biostatistics) and colleagues Zainab Samaan and Ji Cheng explain why this is important and what the benefits are to science, researchers, healthcare providers, funders and patients.

Pilot studies are investigations proposed to direct the conduct of a main study by assessing the potential of running the main study through investigating the feasibility, the study process, resources needed, methods and potential obstacles to completing the main study.

The significance of pilot studies is demonstrated by the experience gained from conducting the pilot, the results seen and the processes tested. Ultimately the goal of pilot studies is to enhance the likelihood of success of the main study by avoiding pitfalls and ensuring the main study is possible.

Despite the rationale behind the conduct of pilot studies, their uptake and reporting remain suboptimal with many published studies reporting on hypothesis testing rather than the intended pilot studies’ objectives. Many scientific journals were not interested in pilot studies and focused their publications on the results of the main studies. In this part of the blog we provide the rationale for two key aspects of pilot studies:

  1. publishing the pilot study protocol prospectively and;
  2. reporting of the pilot study results.

The importance of publishing pilot/feasibility study protocol

A pilot study protocol is an important step in the planning of the main study and to the rigor of the study. The pilot study objectives should be clearly stated in the protocol.

A pilot study protocol is an important step in the planning of the main study and to the rigor of the study. The pilot study objectives should be clearly stated in the protocol.

Such objectives may be grouped based on: process, resources, management and scientific domains. A protocol may include objectives related to the process such as the testing of research instruments, the recruitment potential, eligibility criteria, establishing sampling frame, logistic problems, and finalizing the main study protocol.

The protocol may also identify specific resources including staffing needed and cost of running the full study to prepare for funding applications; while management issues may be identified in the protocol to address data quality, completion, data analysis methods, training of research staff and infrastructure such as information technology needs.

Scientific objectives may include refining the study questions, examining data variability, study validity, and identify important outcomes to estimate sample size of the main study.

There are several reasons for advocating the publication of pilot protocols:

  1. Avoid selective reporting the results.
  2. Ensure consistency. Studies have shown discrepancy between the unpublished protocol accessed through trial registration or ethics/granting applications and published objectives, methods, and results.
  3. Increase transparency. The study protocol is designed before the commencement of the study and includes details of the planned study.
  4. Improve the methodological quality of the main study.
  5. Assist in forming informed decisions about healthcare based on clinical research.
  6. Accountability. In designing studies, obtaining ethics approval, funding and regulatory adherence, investigators are accountable for these agencies and study participants who signed consents based on the protocol.
  7. Research collaboration. Publishing the pilot protocol will enhance communication among groups of researchers interested in the same area and potential collaborations that may improve recruitment, expansion of the study and avoid doubling the research efforts.

The importance of publishing pilot/feasibility study results

Pilot study results are often misreported as hypothesis testing despite the lack of adequate power to provide such an outcome especially in drug trials.

Pilot study results are often misreported as hypothesis testing despite the lack of adequate power to provide such an outcome especially in drug trials.

Several methodologists have provided a framework for reporting pilot studies, usually in the context of randomized trials however the framework can serve other studies’ pilot design and reporting.

There are several reasons for publishing the results of pilot or feasibility studies, as the results can be:

  1. Of benefit to other researchers in designing or planning to do similar studies. This is particularly important for those studies that show ‘negative’ results regarding feasibility—to learn from the lessons gained.
  2. Useful in evidence synthesis on feasibility. To advance science, we rely on transparent reporting of all studies including feasibility studies.
  3. Important in advancing the transparency and completeness of reporting of science. Like all clinical studies, pilot or feasibility studies involving interventional research must be registered. Publishing the results is the last step in fulfilling the scientific obligation of transparent and complete reporting of every scientific endeavor.
  4. Important for guiding establishment of funding programs. The information on feasibility can be useful to research funding organizations in their decision-making for establishing funding programs.
  5. Important for teaching and learning about pilot/feasibility studies. Published reports of pilot or feasibility studies provide a good resource to use as examples of best or poor practices by teachers, learners, reporting guideline developers, authors, and journal reviewers.
  6. Useful to stakeholders including study participants, patients, and healthcare providers.In many cases, published study reports are the only source through which study participants can learn about the findings from studies that they participated in. Patients can learn about planned studies whose feasibility was confirmed or challenged by pilot/feasibility findings. Similarly, healthcare providers can direct their patients to main planned interventional studies.

Pilot studies are important steps in planning and conducting rigorous studies ensuring transparency and accountability. An African proverb says “If you want to go fast, go alone. If you want to go far, go together.”

Publishing both the protocol and the results of pilot or feasibility investigations allows the research community to learn from each other’s mistakes and feedback—that’s how we advance science and societal health through journeying together!

View the latest posts on the On Medicine homepage