Systematic Reviews

A protocol is a detailed document that describes the research plan before the systematic review even begins. It specifies the objectives, methods, inclusion/exclusion criteria, databases to be searched, and the planned analysis steps. A protocol ensures transparency and reduces the risk of bias by preventing methods from being changed as the review progresses.

Why write a protocol?

• Planning: A protocol serves as a roadmap to keep the team aligned, support task assignment, and reduce bias.
• Standards compliance: It fulfills the requirements of PRISMA and the Cochrane Handbook, facilitating manuscript writing and aligning team expectations.
• Efficiency: By providing structure and guidance, the protocol simplifies the review process, facilitates data extraction, and saves time when drafting the final report.
• Bias reduction: By defining criteria in advance, you minimize bias and can track any modifications made during the review.
• Consistency: Clear procedures help prevent errors during study selection and data extraction.
• Transparency and reproducibility: A detailed protocol enables others to reproduce your study, strengthening confidence in your results.
• Quality assurance: A well-structured protocol improves the chances of publication by ensuring a high-quality review.
• Reduced arbitrariness: It provides a structured plan that limits improvised decisions.
• Prevention of duplication: A protocol helps avoid redundant research by identifying areas already studied.
• Anticipation of challenges: It helps anticipate potential obstacles, such as data gaps or complex inclusion criteria.
• Requirements: Protocol registration is often recommended by journals and funders, which increases project transparency.

Par exemple, les protocoles de revues systématiques peuvent être enregistrés dans PROSPERO, le registre international prospectif des revues systématiques.

Reference

• Covidence (2024). A Practical Guide: Protocol Development for Systematic Reviews.

There are many standards that specify the required information for drafting a protocol; following one ensures that your systematic review is methodologically rigorous, transparent, and reproducible. Requirements may vary depending on where you intend to publish your protocol; if you already know, check their specific guidelines.

Most of these standards, however, include some common sections:

• Title and registration: Use a clear, descriptive title that includes the word protocol to signal the planned nature of the study. Provide a registration number if available, to distinguish the protocol from completed reviews and to indicate that it is an ongoing project.
• Introduction, rationale, and objectives: Provide an overview of the problem and justify the need for the review. Clearly state the research question the review aims to address, often using a framework such as PICO.
• Eligibility criteria: Define the inclusion and exclusion criteria based on population, intervention, language, date, publication status, etc.
• Information sources and search strategies: Specify the databases and other sources consulted, including grey literature. Provide a complete search strategy, showing keywords, search operators, and limits (e.g., language, date). Indicate the planned or actual date of the search and consider rerunning the search if the selection process extends over several months.
• Developing the search strategy: Pilot test the strategy to ensure a balance between sensitivity and specificity, revising it as necessary. Document and justify any future modifications, such as adding new keywords as new literature emerges.
• Study selection process: Describe the study selection process, duplicate management, and whether screening of titles, abstracts, and full texts will be performed by two reviewers. Mention the software used, such as Covidence, or reference managers like EndNote or Zotero, and document the process with a PRISMA flow diagram.
• Data extraction: Specify what data will be extracted from each study and provide an initial extraction template, with plans to refine it after pilot testing.
• Assessment of bias or quality: Describe the tools to be used for assessing study quality.
• Data synthesis and analysis: Describe the planned synthesis methods, which may include narrative synthesis, sensitivity analyses, subgroup analyses, or meta-analysis.
• Funding and conflicts of interest: State sources of funding and disclose any potential conflicts of interest.
• Amendments: Document any changes to the protocol and update the registration record accordingly.

Optional sections may also be useful:

• Project timeline: Setting out a timeline for the different stages of the project is particularly helpful for larger teams.
• Dissemination plan: Indicate how you intend to share your results, whether through peer-reviewed publications or by sharing them on platforms such as the Open Science Framework (OSF).

Practical Tips for Writing a Protocol:

• Use the future tense.
• Prefer the active voice (“we will review the titles and abstracts”) rather than the passive (“titles and abstracts will be reviewed”) for a more direct and engaging style.
• Use accessible language: not all readers of systematic reviews are academics or experts, so avoid unnecessary jargon (consider your audience and adapt vocabulary).
• Structure: it is advisable to write the protocol in complete sentences for clarity and to facilitate understanding by others.

Amendments: Yes, But…

A systematic review is an iterative process, which means it is always possible to make modifications to the protocol if necessary. However, modifying the protocol after data extraction and quality assessment may introduce potential bias. Therefore, it is recommended to make changes only before the data extraction stage.

Examples of Protocol Guidelines:

• Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P)
• Campbell Systematic Reviews: Policies and Guidelines
  • Chapter 5: Review protocol
• JBI Systematic & Scoping Reviews: Guidance for Protocols
• Evidence Synthesis Protocol Templates (Ghezzi-Kopel & Porciello, 2020)
• Generalized Systematic Review Registration Form (Van den Akker, O. et al., 2020)
• Inclusive Systematic Review Registration Form (Van den Akker, O. et al., 2023)
• An integrative framework for planning and conducting Non-Intervention, Reproducible, and Open Systematic Reviews (NIRO-SR) (Topor, M. K. et al., 2023)

Staying Up to Date with Guidelines:

• Subscribe to relevant email listservs to receive updates, discussions, and announcements.
• Check the EQUATOR Network, an international initiative to improve transparency and quality of research reporting by providing resources, guidelines, and tools.
• Participate in workshops and webinars offered by the scientific community.

Reference

• Covidence (2024). A Practical Guide : Protocol Development for Systematic Reviews.

Protocol registries are databases or platforms where researchers register their study protocols, particularly for systematic reviews and meta-analyses, before starting their research.

This pre-registration process aims to increase transparency, accountability, and reproducibility in research by making the study objectives, methods, and planned analyses publicly accessible. Protocol registries ensure that researchers follow the pre-registered methods, thereby reducing the risk of bias related to selective outcome reporting or mid-project changes that could alter the conclusions. They also make it possible to track studies and prevent duplication.

Here are some examples of protocol registries:

• Campbell Collaboration – Focuses on protocols for systematic reviews in the social sciences, including education, crime, and justice.
• Cochrane Protocols – For health sciences, specifically for systematic reviews published in the Cochrane Database.
• Open Science Framework (OSF) – Supports protocol registration in different disciplines, allowing researchers to share their protocols and other research materials.
• PROCEED (Collaboration for Environmental Evidence) – Designed to archive and provide access to protocols and systematic reviews related to environmental sciences.
• PROSPERO – Mainly for systematic reviews in health.
• Protocols.io – An open-access platform where researchers can create, share, and publish detailed step-by-step scientific protocols, promoting collaboration, reproducibility, and transparency.
• Research Registry

Best practices

• Clear role assignment: At the beginning of the review project, clearly define roles and expectations for each team member. Identify the project leader, methodologist, statistician, librarian, information specialist, etc. Specify how many people are needed for each stage (e.g., title and abstract screening, full-text review, and data extraction). Include the initials of lead reviewers in the protocol to clarify responsibilities, including who will resolve conflicts.

• Flexibility for revisions: Because the review process is iterative, it is normal to revise and adjust the protocol before final registration. Discuss timelines with the team and, if necessary, conduct pilot testing of screening and data extraction steps, adjusting the protocol as needed. This flexibility ensures the protocol accurately reflects the team’s needs.

• Acceptance and documentation of changes: Record all changes in the protocol registry, thereby strengthening transparency and credibility. Modifications to the protocol, especially after public registration, should be noted and justified along the way.

• Piloting templates: Test templates for data extraction and quality assessment to verify that they capture the relevant information. Pilot testing allows the team to refine templates based on feedback from initial analyses, improving the quality and consistency of data collection.

Common pitfalls

• Lack of a protocol

• Vague objectives: Objectives that are too broad or imprecise from the outset can create confusion. Clearly define the research question to ensure it aligns with the scope of the review.

• Unclear eligibility criteria: Vague inclusion and exclusion criteria can result in inconsistent study selection. Clearly define elements such as population, interventions, comparators, outcomes, and study types to ensure uniform decisions during selection.

• Inadequate search strategy: An insufficient search risks missing important studies. Develop a comprehensive strategy in collaboration with a librarian or information specialist to ensure all relevant aspects are covered.

• Insufficient time for writing: Rushing the writing of the protocol can compromise its quality. Allocate sufficient time to ensure the document is rigorous and accurate.

• Neglecting data extraction details and bias assessment: Document your methods for data extraction and quality assessment at the outset, especially when multiple types of studies are included.

• Lack of collaboration: Poor collaboration and unclear roles within the team can lead to arbitrary decisions. Define roles at the beginning and include them in the protocol to ensure balanced contributions from all members.

• Failure to register: Not registering the protocol limits visibility, increases the risk of duplication, and can lead to wasted effort. Register it in advance to promote transparency. Some journals also publish protocols, which increases visibility and reduces the likelihood that other researchers will work on the same topic without knowing.

Reference

• Covidence (2024). A Practical Guide : Protocol Development for Systematic Reviews.

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement

• Moher, David, Larissa Shamseer, Mike Clarke, Davina Ghersi, Alessandro Liberati, Mark Petticrew et al. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement, Systematic Reviews, vol. 4, no 1. Récupéré de https://doi.org/10.1186/2046-4053-4-1