This guide is based on ongoing testing of multiple AI and automation tools by experienced methodologists. AI can help increase efficiency and automate certain tasks in evidence synthesis — but it cannot replace human judgment or oversight. These findings align with the recent Artificial Intelligence (AI) Methods in Evidence Synthesis webinar series by Cochrane. We share this resource to support responsible exploration of AI tools and workflows. Use with care, transparency, and critical thinking.
Workflow Stages & Potential Tools |
|||
|---|---|---|---|
Plan |
Identify |
Extract & Evaluate |
Combine, Summarize &
|
|
Frame research questions |
Generate search terms |
Auto-extract from PDFs |
Meta-analysis |
|
Summarize the literature |
Citation searching |
RCT evaluation |
Writing assistants |
|
Project/Meeting notes |
Screening/Deduplication *uses active learning for study ranking and auto-deduplicates |
Bias-assessment visualization |
Multi-function (search, summarize, report) |
They can help find synonyms, related organizations, conference names, and terms in other languages. However:
Some tools (e.g., Elicit) claim to handle the full review process. In practice, results can vary widely—even with identical prompts—and important studies may be missed (Bernard et al., 2025).
LLMs can help researchers translate a broad interest into a structured research question using common frameworks like PICO (Population, Intervention, Comparator, Outcome), PEO (Population, Exposure, Outcome), or PCC (Population, Concept, Context).
A public health researcher wants to study the impact of urban green spaces on mental health. They prompt an LLM with:
The model suggests:
This gives the researcher a structured starting point, but the researcher must adapt it by refining the population, clarifying exposures, prioritizing outcomes, and aligning with project scope.
The prompting process is iterative. If the LLM’s response is vague or off-track, add more context (e.g., specify age groups or study designs) or rephrase the request.
AI can accelerate idea generation, but the final framing requires human expertise for accuracy and reproducibility.
Objective: Use Google NotebookLM (or a GPT-based tool) to generate a comparative table summarizing key aspects of review articles.
1. Log in to Google NotebookLM using your institutional NetID and password.
2. Add content by uploading article PDFs.
3. Enter this prompt:
4. Review the AI - generated table for accuracy and missing data. Revise the prompt if needed to improve clarity or add context.
5. Click "Save to Note" of you want to keep the output in your Notebook.
Objective: Use a conversational AI (Gemini or Copilot Chat) to brainstorm search terms for a database search.
1. Open your preferred AI tool (Gemini, Copilot, ChatGPT, etc.).
2. Enter your prompt. Here is an example:
3. Review the output. Look for:
4. Refine or follow up with additional prompts like:
Objective: Analyze MeSH terms across multiple relevant articles using the Yale MeSH Analyzer (an automation tool).
1. Go to Yale MeSH Analyzer.
2. Enter your PubMed IDs into the input box. Here are sample IDs to experiment with:
3. Click “Go.”
4. Examine the resulting table, which includes: Article titles, MeSH terms, and more.
Note: To display abstracts, select this option on the search page.
5. Reflect:
A similar activity can be done using PubReminer which generates frequency tables for keywords and MeSH terms.
