Systematic Reviews

As with any research, it is essential to clearly define the objective by developing a well-formulated question. A good systematic review relies on a precise and answerable question. It is an iterative process, moving from the initial question to the formulation of the final question.

• A well-structured question increases the likelihood of obtaining relevant results.
• It is essential to formulate a research question carefully in order to avoid overlooking relevant studies or gathering a potentially biased set of results.

Defining the Question

This is an important step in the systematic review process since the question shapes many aspects of the work. It:

• serves as a guide to communicate the overall objective of the review;
• helps to establish eligibility criteria;
• sets boundaries for deciding which studies will be included in the review;
• determines which data should be extracted from each study;
• structures the presentation of results.

(Evidence Synthesis Institute, 2024)

Iterative Process

• Identifying the topic of interest
  The starting point is to identify a general area or topic of interest. This may come from an observation, a gap in the literature, or a question raised in your discipline.

• Refining the initial question
  From the general topic, a first research question is formulated. This question may be broad at first and then refined to make it more precise and focused.

• Preliminary literature search
  A rapid review of existing studies helps to assess the current state of knowledge. This helps identify gaps in the literature and specify what should be addressed in your question.

• Clarifying key variables
  At this stage, you clearly define the key variables (e.g., population, interventions, comparators, outcomes) to give the question a more rigorous structure.

• Refining inclusion and exclusion criteria
  By refining eligibility criteria (which studies to include or exclude), you further adjust the scope of the question to make it realistic and applicable to a systematic review.

• Feasibility check
  It is important to verify whether the research question is feasible—i.e., whether enough relevant studies exist to allow a systematic review and whether the necessary data are accessible.

• Revision and final adjustment
  Once all these elements are considered, the research question is finalized. This process may require several iterations to ensure the question is clear, precise, and feasible within the scope of your review.

Here are some questions to ask yourself to assess whether your research question is well-formulated and appropriate:

• Importance of the problem: Is this problem important or urgent enough to require a solution?
• Context and prior assessment: Have previous evidence syntheses already been conducted? Did they reveal the need for new studies?
• Information yield: Is the proposed study broad enough to provide meaningful information?
• Pragmatism: Does the research reflect real-world conditions? If not, does it matter?
• Patient-centeredness: Does this research reflect patient priorities?
• Value for resources: Is this research worth the investment in terms of resources?
• Feasibility: Can this research realistically be conducted?
• Transparency: Are the methods, data, and analyses verifiable and free from bias?

(Ioannidis, 2016)

Modifying the Research Question after Data Collection: Some Precautions

Although refinement of the question is possible, it is important to guard against bias when modifying the question. Changing a question based on data may lead to misleading conclusions. Any modification resulting from a re-evaluation of the review question should be documented.

Questions to ask when refining the research question:

• What is the motivation for this refinement?
• Could this refinement have been influenced by the results of one of the included studies?
• Are the search strategies appropriate for the refined question (particularly those already implemented)?

(Cochrane Handbook, 2024)

References

• Evidence Synthesis Institute (2024). « Formulating a Research Question », présentée à la Evidence Synthesis Institute Conference.
• Higgins, Julian, James Thomas, Jaqueline Chandler, Miranda Cumpston, Tianjing Li, Matthew Page et al (dir.) (2024). Cochrane Handbook for Systematic Reviews of Interventions version 6.5 (mis à jour en août 2024), Cochrane. Récupéré de https://training.cochrane.org/handbook/current
• Ioannidis, John P. A. (2016). « Why Most Clinical Research Is Not Useful », PLoS Med, vol. 13, no 6. https://doi.org/10.1371/journal.pmed.1002049

Formulating a well-constructed research question is essential for a successful review. You should have a draft research question before choosing the type of review you will conduct, as this will help guide you toward the most appropriate type.

Systematic Review

• Generally based on a focused research question with narrowly defined parameters.

Scoping Review

• Often based on a broad research question that aims to address wider, more complex, and exploratory issues.

Narrative Literature Review

• Typically addresses a less structured question, allowing exploration of a topic in a broader way by integrating different perspectives and providing an overview of existing knowledge without necessarily applying strict criteria.

To determine which type of knowledge synthesis best suits your project, refer to the decision tree presented earlier.

Frameworks are used to break down the research topic into key concepts and to develop the research question. They serve as a reminder of the elements of interest and help identify: how, who, what, when, and where.

Several frameworks can help you structure your research topic into fundamental concepts:

Population / Intervention, Exposure / Comparison / Outcome

• Mainly used in medicine; also applies to other disciplines.
• Commonly used for systematic reviews assessing the effectiveness of an intervention.

This is the reference framework used by the Covidence platform, which aims to simplify the screening and selection of articles in systematic reviews.

Sample (who) / Phenomenon of Interest (what) / Design (how) / Evaluation (compared to) / Research type (which types)

• Used in social sciences and qualitative research for studies on participants’ experiences and the evaluation of interventions or social phenomena.
• Suitable for qualitative reviews focusing on more subjective aspects, such as perceptions or experiences.

Expectation (why) / Client group (for whom) / Location (where) / Impact (measured how) / Professionals (delivered by) / Service (what)

• Used in public health, health service management, and public policy to evaluate interventions.

Setting (where) / Perspective (for whom) / Intervention (what else) / Comparison (compared to) / Evaluation (with what results)

• Used in social sciences, education, and in implementation reviews of interventions across different contexts.
• Suitable for qualitative or policy evaluation reviews.

Population / Exposure / Outcomes (themes)

• Mainly used in social sciences and for qualitative or epidemiological reviews.
• Appropriate for qualitative studies examining exposures and their outcomes in a given context.

Context / Intervention / Mechanisms / Outcomes

• Used in program research and impact evaluations.
• Mainly used in realist reviews, which explore how and why an intervention works in a given context, and studies aimed at understanding the mechanisms behind how an intervention functions within a particular context, especially in public health, policy evaluation, and administration.

You can consult this rapid overview of existing research question frameworks.

What if my topic does not fit into a framework?

Not all topics fit neatly into a framework. If this is the case for your topic, use only the framework elements that are relevant.

Inclusion and Exclusion Criteria

Inclusion and exclusion criteria form the basis for determining whether sources are selected or excluded in a systematic review:

• Inclusion criteria are all the elements a study must have in order to be included in your review;
• Exclusion criteria are the factors that would make a study ineligible for inclusion in your review.

They are fundamental for screening and guiding the review process. Criteria must be clearly defined. They are generally established after formulating the research question and before the final search, although preliminary searches may be needed to determine appropriate criteria. This process may also lead to revisions of the research question if necessary.

Consider how location, study methodology, and language might influence your review:

• Which criteria are of particular importance?
• Does a criterion introduce an unacceptable level of bias?

Examples of criteria to consider include:

• Participants or subjects;
• Intervention;
• Study design;
• Sampling method;
• Outcome measures;
• Language;
• Publication type;
• Context;
• Date of publication;
• Geographic location;
• Etc.

Eligibility criteria define the foundation on which the article selection will be based.

Reference 

• Evidence Synthesis Institute (2024). « Question Framework and Eligibility Criteria », présentée à la Evidence Synthesis Institute Conference.