Skip to Main Content

Systematic Reviews

This guide explains systematic reviews and provides tools, strategies, and library resources to help you plan and carry out your research.

Search Strategy

The quality of a systematic review depends on its search strategy.

To carry out a systematic review, it is essential to obtain robust and reproducible search results. Documenting all decisions—including what worked and what did not—is crucial, particularly for future updates.

According to the MECIR Guidelines (2023):

  • Search strategies for systematic reviews should be as comprehensive as possible to minimize publication bias and to capture all relevant data (C24);
  • Search strategies should be carefully tailored to each database to avoid missing important data, and should be iteratively refined based on search results (C33);
  • The entire search process—including databases used, search terms, dates, and personnel responsible—must be thoroughly documented to ensure full transparency and reproducibility (C36);
  • To keep the review up to date, searches should be rerun within 12 months (preferably 6 months) before publication, with newly identified studies incorporated into the review (C38).

Practical tips for developing your search strategy:

  • Save your search strategy frequently (e.g., within the database account and in a tracking document).
  • Review your strategy carefully (step back and revisit it a few days later).
  • Pay close attention to the number of results; errors can sometimes be detected by checking whether the number of results aligns with expectations based on the search logic.

Steps in Developing a Search Strategy

Identify concepts and develop keywords

Use the protocol or methodological framework as a guide:

  • Focus on the essential key concepts.
  • Avoid including inclusion or exclusion criteria, since some are better applied during the study selection process.
  • Think about concepts likely to appear in titles and abstracts, rather than only in the full text.
  • Find a balance between having too few concepts or too many.

Develop the search in the main database

  • Generally start with the database most likely to return the largest number of results, which helps estimate the overall number of studies to review.
  • This database is also likely to contain the most relevant thesaurus terms.

Identify appropriate controlled vocabulary and free-text terms

Controlled vocabulary: indexing terms from a database thesaurus

  • Where to find indexing terms:
    • Check the database thesaurus (fr).
    • Look at your reference articles (locate them in a database and check their indexing).
  • Note:
    • New or emerging subjects, as well as topics that are too specific or under-documented, may not yet have indexing terms.
    • Some databases do not use a thesaurus.

Free-text keywords: terms used by authors in their manuscripts

  • Start with the keywords your team already knows.
  • Check the titles, abstracts, and keywords of reference articles on your topic.
  • Consult dictionaries and encyclopedias to become familiar with the terminology of your subject.
  • Consider spelling variations, synonyms, antonyms, acronyms, regional terms, terms that have changed over time, truncation, and proximity operators (these allow you to search for multi-word concepts while keeping some flexibility, unlike exact phrase searching).

Sensitivity and precision in searching

  • Is your search sensitive enough to capture most relevant results? A sensitive search retrieves the majority of relevant studies but may also include more irrelevant ones.
  • Is your search specific enough to reduce “noise”? A specific search reduces irrelevant results but risks missing some relevant ones.

Testing the sensitivity of a search

  • The goal of a systematic review is to be comprehensive and exhaustive. To increase the sensitivity of your search, consider using both thesaurus indexing terms and free-text keywords.
  • Use reference articles (those you know will be included in your review) to test your search strategy.

Search operators, proximity operators, and search features allow you to structure queries for a systematic review. They help expand or narrow results, increasing the precision and sensitivity of the search to identify relevant studies.

Search Operators

AND – Used to narrow the search by combining ideas or concepts (keywords)

  • climate change AND businesses

OR – Used to broaden the search by linking keywords related to the same concept (synonyms, related terms, translated terms)

  • businesses OR companies

NOT – Used to exclude a keyword from the search results

  • businesses NOT SMEs

Proximity Operators*

Used to combine ideas or concepts (keywords) by limiting the number of words between two keywords or ideas.

Proximity

  • PROX/n / NEAR/n – n represents the maximum number of words between the two keywords. Proximity operators retrieve terms regardless of order.
    • businesses PROX/5 multinationals

Adjacency

  • ADJ/n / PRE/n – n represents the maximum number of words between the two keywords. Adjacency respects word order.
    • climate ADJ/2 change

*Check the database search help section: not all databases support proximity operators, and the syntax may vary between databases.

Search Features

Truncation * – Used to search for the root of a word and its different endings

  • environ* = environment, environmental, environments

Quotation marks "..." – Used to search for an exact phrase

  • "climate change"

Parentheses ( ... ) – Used to prioritize part of the query

  • climate AND (businesses OR companies)

Wildcard ? – Replaces 0 or 1 character within a word (multiple wildcards can replace multiple characters)

  • labo?r = labor or labour
  • wom?n = woman or women

Note: the symbol and usage may vary depending on the database.

Search strategies for studies must be as comprehensive as possible to minimize the risk of publication bias and capture the maximum amount of relevant evidence. This requires consulting:

  • General databases (e.g., Web of Science, ProQuest Central, EBSCO Academic Search Complete);
  • Subject-specific databases (e.g., PsycINFO, ERIC).

For a systematic review, it is essential to select multiple databases to ensure complete coverage of the available evidence.

Why use multiple databases?

  • Ensure comprehensive coverage of the topic;
  • Capture interdisciplinary subjects;
  • Minimize bias;
  • Access grey literature;
  • Respond to researchers’ preferences;
  • Cover a broader geographical scope.

When selecting a database for a systematic review, several elements should be considered to ensure a relevant and comprehensive search.

Subject coverage:

  • Multidisciplinary: For cross-disciplinary approaches (e.g., Web of Science Core Collection, Academic Search Complete);
  • Specialized: For specific fields (e.g., MEDLINE, APA PsycINFO);
  • Highly specialized: For very narrow or advanced topics.

Geographical coverage: Choose databases that cover the regions or countries relevant to your research.

Database content:

  • Coverage periods: Verify the years included in the database;
  • Desired publication types: Include theses, government documents, grey literature, etc.;
  • Publication types to exclude: For example, trade magazines or news items;
  • Coverage of specific journals and available languages.

Search features:

  • Availability of controlled vocabulary (descriptors) to improve precision;
  • Use of Boolean and proximity operators;
  • Ease of combining result sets;
  • Advanced functions, such as field searching and command-line searching.

Technical aspects of the database:

  • Access platform and ability to save searches and search history to ensure reproducibility;
  • Result management: tools for detecting duplicates, download limits, and export formats;
  • Download limitations and format availability.

To make an informed choice of databases, consider these tips:

  • Run a quick search: Browse the results to check their relevance to your research question;
  • Search across multiple databases: This helps identify which ones are most suitable for your topic;
  • Review other systematic reviews on similar topics to see which databases were used;
  • Consult library guides and database lists from other institutions to broaden your options.

Finalize the strategy in a key database

The strategy developed in your key database should be reported in the protocol and used to create strategies in other databases. It is essential to organize and record the strategy in a logical and readable way. The final search strategy can be reported line by line, in blocks, or in a single string, depending on the search interface.

Adapting the search strategy

When adapting the search strategy across databases, you should:

  • Adjust subject headings according to each database’s controlled vocabulary;
  • Translate the search syntax to account for database-specific requirements (e.g., proximity operators, search features, fields);
  • Keep a search log and save strategies in database accounts.

Some tools can facilitate this process:

  • Polyglot – helps automatically translate search queries from one database to another, adapting to each platform’s syntax;
  • Search Refiner – assists in refining and adapting search strategies across databases by providing suggestions.

Limits (or filters) are built-in options in a database that allow you to restrict search results to certain criteria without modifying the search strategy itself. For example, you can limit results by selecting:

  • Peer-reviewed articles (to minimize risk of error and bias);
  • Article language (e.g., only English or French);
  • Publication type (e.g., reviews, clinical trials);
  • Publication date range;
  • Geographic region;
  • Etc.

Limits can quickly reduce the volume of results, but relying on them exclusively may lead to the loss of relevant documents, especially if the database has not indexed some content correctly.

Search filters (sometimes also called “search hedges” or “search blocks”) are predefined search strategies. They consist of sets of search terms (keywords, phrases, indexing terms) developed for a specific concept (such as a methodology or a population). Search filters are designed to ensure more comprehensive coverage of a topic and can be incorporated directly into a search strategy to refine results.

If you reuse search filters developed by others, please cite or acknowledge them.

Here are some examples of search filter resources:

Things to consider before using search filters:

  • Is it up to date?
    • Has the database syntax changed since it was created?
    • Have subject headings evolved? Could some new descriptors be missing?
    • Have keywords changed over time?
  • Was it designed for specificity or for sensitivity (and which matters more for your research)?
  • Was it designed for a search interface to which you currently have access?

Reference

Evidence Synthesis Institute (2024). « Wirting a Search Strategy and Translating it to Multiple Databases », présentée à la Evidence Synthesis Institute Conference.

Exploratory Search

Exploratory searching is an essential preliminary step for a systematic review, aiming to better understand the nature of the topic.

It is a flexible, learning-focused process designed to build a nuanced understanding of a subject, often drawing on diverse and sometimes contradictory sources. This type of search is less focused on precision and more on active navigation and continuous adaptation, allowing researchers to develop and refine their understanding of the topic as they go.

Key features of exploratory searching:

  • Abstract objective: The goal is broad—generally to understand the scope and context of the topic rather than to locate specific documents.
  • Iterative process: Researchers continuously adjust their approach as new information emerges during the search.
  • Initial broad queries: Searches often begin with broad queries to become familiar with available sources.
  • Responsive navigation: Researchers actively explore while passively collecting cues in the search environment, which guide their next steps.
  • Adapting strategies: Strategies and objectives are refined throughout the session as new information is assimilated.

(Gusenbauer & Haddaway, 2021)

Gathering a sample of articles

  • Ask your team for known articles that “should be included” in the review.
  • Identify your own articles: explore Google Scholar and one or two specialized databases.
  • Collect a set of varied yet representative references that meet your inclusion criteria:
    • Different publication dates;
    • Different disciplines;
    • Different study designs or methodologies;
    • Different geographic regions.

It is essential to define your target clearly in order to guide your search effectively and ensure the relevance of your results.

Questions to ask yourself when exploring your article sample in databases:

  • Which terms do authors use in titles and abstracts? Use their terminology to build your vocabulary.
  • For databases with a thesaurus, check your seed articles and review the assigned subject headings. Are they consistent, specific enough, and available?
  • Are there author-assigned keywords?
  • Are certain authors or articles cited multiple times in your sample? This may help you identify reference studies or key publications to include in the review.

Here are the main objectives that exploratory searching should achieve:

  • Confirm or refine the research concepts;
  • Identify an initial list of keywords;
  • Identify an initial list of subject headings in one or two databases;
  • Assemble a validation sample of articles to test your searches.

 

Tip: Document exploratory searches carefully, as they provide valuable insights for refining your final search strategy.

Reference

Gusenbauer, Michael et Neal R. Haddaway (2021). What every researcher should know about searching–clarified concepts, search advice, and an agenda to improve finding in academia, Research synthesis methods, vol. 12, no 2, p. 136-147. https://doi.org/10.1002/jrsm.1457 

Adjusting the search strategy is an iterative process, where each preliminary attempt helps refine and optimize the query in order to comprehensively capture the relevant evidence. This process involves:

  • Ongoing evaluation of results to refine terms and operators used;
  • Adapting strategies to the specific features of each database;
  • Incorporating feedback and insights from relevant articles to adjust search criteria and maximize coverage of the evidence.

Each adjustment should also be carefully documented to ensure full transparency and reproducibility.

Exporting Records

To export references efficiently from databases for a systematic review, it is recommended to follow a few best practices:

  • Create a user account: In many databases (such as ProQuest), creating an account increases export limits.

  • Use the appropriate export format: For smooth transfer to reference management software (e.g., EndNote, Zotero) or to Covidence, it is recommended to export references in RIS format.

  • Export in batches: When handling a large number of references, it may be necessary to export in several batches to avoid database-imposed limits.

  • Include essential metadata: Make sure your export includes key fields such as keywords, abstracts, and DOI identifiers, which will facilitate screening and access to full texts.

To learn more about these tools, consult the following guides:

A workflow that integrates Covidence with reference management software is presented in the study selection stage.

Google Scholar

In the context of a systematic review, the use of Google Scholar can present challenges due to its limited reproducibility and lack of transparency.

Challenges of using Google Scholar for a systematic review:

  • Reproducibility issues: Results may vary depending on the user’s search history, making searches difficult to reproduce accurately (using incognito mode may help).
  • Lack of transparency: Google Scholar does not clearly disclose its indexing criteria or search algorithms.
  • No controlled vocabulary: There are no standardized subject terms or thesauri to refine searches with precision.
  • Limited advanced search: The advanced search interface is not precise enough for detailed systematic searches.
  • Few filtering options: Limited ability to filter by publication date, discipline, or other specific criteria.
  • No sorting options: Results cannot be sorted by date, relevance, or other parameters.
  • Ranking bias: Older, more frequently cited articles are prioritized, potentially overshadowing more recent research.
  • Results cap: Searches are limited to 1,000 results, which restricts comprehensive coverage.
  • Limited export options: No bulk export function for citations, making data extraction time-consuming.

How Google Scholar can still support systematic review projects:

  • Preliminary searching: Useful for gathering general information, exploring terminology, and identifying studies to test search strategies in other databases.
  • Access to grey literature: Provides access to less conventional sources and reports not indexed in traditional databases.
  • Backward citation searching: Allows you to follow citations of known studies to track the development of research.
  • Access to full texts: Helpful for locating the full text of known articles.
  • Complementary tool: Complements other database searches by adding an additional layer of literature not always captured in traditional academic databases.

Exporting from Google Scholar

The export functions in Google Scholar are limited, particularly because it is not possible to export multiple references in bulk. However, there are some useful alternatives:

Zotero Connector

The Zotero Connector allows you to export all the results from a search results page in one click, making it easier to collect and organize many references for research projects.

Publish or Perish

The Publish or Perish software can be used to bulk export references from Google Scholar into a RIS file, which can then be imported into bibliographic management tools such as EndNote or Zotero.

Here is an overview of the steps to follow:

  • Search: Use Publish or Perish to search for relevant articles in Google Scholar.
  • Select results: Choose the number of results you wish to export.
  • Export: Save the results in RIS format.
  • Import: Import the RIS file into EndNote or Zotero.

Order of importing references

It is recommended to import references from the main databases first, and then add those from Google Scholar. Metadata provided by Google Scholar is often less complete.

In Covidence and EndNote, the deduplication algorithms tend to keep the first version of a reference, which is usually more complete if it comes from the main databases.

In Zotero, however, duplicates can be merged manually, allowing you to retain the most detailed information from each reference.

Grey Literature

What is grey literature?

Grey literature refers to information produced by individuals or organizations outside traditional publishing and distribution channels (Cochrane Handbook, Chapter 4).

  • Technical reports
  • Conference proceedings
  • Preprints
  • Patents
  • Reports
  • Working papers
  • Theses and dissertations
  • Unpublished and ongoing studies

Why include grey literature?

Systematic reviews should be as comprehensive as possible to reduce the risk of publication bias and to identify the maximum amount of relevant evidence (MECIR, Standard C28).

  • A large share of conference proceedings are never published as journal articles, a proportion that varies considerably by discipline and the nature of the conference.
  • Conference proceedings and other types of grey literature represent about 10% of references in Cochrane reviews (Campbell Systematic Review, 2017, p.15).
  • Theses and dissertations often contain rich details.
  • The importance of grey literature varies across disciplines and topics.

How to find grey literature

Searching for grey literature requires creativity and flexibility. There is no single source or method.

Approaches to consider:

  • Go directly to the source:
    • Non-profit or non-governmental organizations
    • Government agencies
    • Professional associations
  • Seek advice from subject experts.
  • Some types of grey literature can be located in databases (conference proceedings, theses and dissertations, some reports).

You may consult the Appendix II of the Campbell Systematic Reviews, which provides a list of grey literature resources.

Documenting grey literature searches

It is essential to document your sources, search methods, and results carefully.

Search
Maintain a log file recording the following:

  • Sources/URLs consulted
  • Search terms used
  • Search approach (e.g., advanced Google search, site search, built-in search function, etc.)
  • Number and relevance of results
  • Date of search and date results were reviewed

Results
Record key bibliographic information, including the access date and the source (e.g., URL). Download full texts and, when possible, include the download date in the file name.

Exporting grey literature

  • Full text: Often accessible directly, although bulk export options are rare.
  • Reference management: Few grey literature sources support direct export into reference management software, making organization more challenging.
  • Manual alternative: Enter references manually into a reference management tool.

References

  • Lefebvre, Carol, Julie Glanville, Simon Briscoe, Robin Featherstone, Anne Littlewood, Maria-Inti Metzendorf et al. (2024). « Searching for and selecting studies », dans Cochrane Handbook for Systematic Reviews of Interventions.
  • Kugley, Shannon, Anne Wade, James Thomas, Quenby Mahood, Anne-Marie Klint Jørgensen, Karianne Hammerstrøm et al. (2017). Searching for studies: a guide to information retrieval for Campbell systematic reviews, Campbell Systematic Reviews, vol. 13, p. 1-73. https://doi.org/10.4073/cmg.2016.1

Other Sources

There are several tools that can help identify and visualize interconnected references to support systematic review projects:

Documenting the Search Strategy

The search process (including sources consulted, dates, people involved, and terms used) must be documented in sufficient detail throughout. This ensures accurate reporting in the review and allows all searches across databases to be reproduced. (C36 from MECIR)

According to the PRISMA-S: PRISMA Search Reporting Extension (translated and adapted), here are examples of information to document:

Information on Sources and Methods

1. Names of databases

  • Cite each database searched, including the platform hosting it.

2. Multi-database searching

  • If multiple databases were searched simultaneously on a single platform, name the platform and list all databases searched.

3. Study registries

  • List all study registries consulted.

4. Online and print resources

  • Describe any online or print sources deliberately searched or consulted (e.g., tables of contents, printed conference proceedings, websites), and how they were accessed.

5. Citation searching

  • Indicate whether cited or citing references were examined, and describe the methods used (e.g., scanning reference lists, using a citation index, creating email alerts for citations to included studies).

6. Contacts

  • Note whether additional studies or data were sought by contacting authors, experts, manufacturers, or other stakeholders.

7. Other methods

  • Describe any additional sources of information or search methods used.

Search Strategies

8. Full search strategies

  • Include the exact search strategies for each database and source, copied and pasted exactly as run.

9. Limits and restrictions

  • State if no limits were used, or describe any limits or restrictions applied (e.g., date range, language, study design) and justify their use.

10. Search filters

  • Indicate if published search filters were used (either unchanged or adapted) and cite the filter(s) applied.

11. Previous work

  • State if search strategies from previous reviews were adapted or reused (in whole or in part), and cite the prior reviews.

12. Updates

  • Describe the methods used to update searches (e.g., rerunning searches, setting up email alerts).

13. Search dates

  • For each search strategy, provide the date of the last search run.

Peer Review

14. Peer review

  • Describe any process of peer review of the search strategy.

Reference Management

15. Total number of references

  • Document the total number of references retrieved from each database and other sources.

16. Deduplication

  • Describe the processes and software used to deduplicate records from multiple searches across databases and sources.

Roles

Expert:

  • May conduct manual searches in journals and on websites
  • Send requests to mailing lists or to known researchers
  • Search the gray literature

Librarian:

  • Translate search strategies for other databases
  • Run searches in databases and export the results
  • Search the gray literature
  • Keep detailed notes on the search process
  • Record updates to the search strategies