What Is a Systematic Review?

When people say a treatment has “evidence from systematic reviews,” they are invoking the top of the evidence hierarchy. Systematic reviews are designed to find and combine all available research on a question, giving a more reliable answer than any single study. But in practice, the quality varies enormously.

How a systematic review works

The process follows a standard sequence [1]:

1. Define the question. What intervention, for whom, compared with what, measuring what outcomes?
2. Search comprehensively. At least two databases, plus hand-searching reference lists and gray literature.
3. Screen and select. Apply pre-defined inclusion criteria, with two reviewers working independently to reduce bias.
4. Assess quality. Rate each included study for risk of bias using a standardized tool.
5. Synthesize. Describe the findings narratively and, if studies are sufficiently similar, combine them statistically in a meta-analysis.
6. Report transparently. Follow the PRISMA checklist [2].

When it becomes a meta-analysis

A meta-analysis pools the numerical results of individual studies to produce a single effect estimate, usually with a confidence interval. This increases statistical power and can detect effects that no single small study could.

But the pooled number can be misleading. If most included studies are small and biased, the meta-analysis produces a precise estimate of a biased effect. The confidence interval gets narrow, which makes the finding look more certain than it is. This is a particular risk in hypnosis research, where most trials are small and blinding is difficult.

What to look for

Not all systematic reviews are equally trustworthy. A commentary on the proliferation of reviews estimated that most published systematic reviews have significant methodological weaknesses [3]. Here is what distinguishes a good one:

Feature	Good sign	Warning sign
Protocol registration	Pre-registered on PROSPERO or similar	No protocol mentioned
Search strategy	Multiple databases, clear terms	Single database or vague description
Study selection	Two independent reviewers	Selection process not described
Risk of bias	Formal tool applied to each study	Quality mentioned but not assessed
Heterogeneity	Explored and discussed	Ignored despite varied studies
Prediction intervals	Reported alongside confidence intervals	Only confidence intervals shown
Funding and conflicts	Declared	Not reported

AMSTAR 2 provides a formal checklist for appraising systematic reviews themselves [5].

Prediction intervals versus confidence intervals

This distinction matters in hypnosis research. A confidence interval tells you the range within which the average effect probably lies. A prediction interval tells you the range of effects you might expect in a new study. When heterogeneity is high, the prediction interval is much wider.

A surgical hypnosis meta-analysis illustrates this well: the pooled effects were positive, but the 95% prediction intervals crossed zero for most outcomes, meaning that a new study might find no benefit at all [6]. This is a more honest representation of uncertainty than the confidence interval alone.

Systematic reviews in hypnosis

Hypnosis has a growing number of systematic reviews, but they inherit the weaknesses of the trials they summarize. Common patterns:

• Small included studies. Many meta-analyses in hypnosis include trials with 20–40 participants each.
• Heterogeneous interventions. Studies combine different hypnotic protocols, delivered by different people, in different settings.
• Weak comparisons. Wait-list and treatment-as-usual controls are common; active comparisons are rare.
• Limited blinding. Most trials cannot blind participants or therapists.

A well-conducted meta-analysis of 85 laboratory pain studies is an example of the form done well: clearly defined inclusion criteria, a formal risk of bias assessment, subgroup analyses by hypnotizability and a transparent discussion of limitations [4].

The hierarchy is not automatic

Placing systematic reviews at the top of the evidence pyramid is useful as a general rule, but it breaks down when the reviews themselves are poorly conducted or when they summarize low-quality primary studies. A single well-designed large trial can be more informative than a meta-analysis of twenty small, biased ones.

What this means for reading hypnosis research

When you encounter a claim supported by “a meta-analysis,” check:

1. How many studies were included, and how large were they?
2. What were the comparison conditions?
3. Was risk of bias formally assessed, and how did it affect the results?
4. Are prediction intervals reported?
5. How heterogeneous were the results?

For a companion guide to reading individual trials, see What Is a Randomized Controlled Trial?. For why the comparison group matters so much, see Why Comparison Groups Matter.

A note on certainty. Neither a systematic review nor a meta-analysis makes something true. They are methods for organizing evidence, and the conclusions they reach are always provisional. When a review says the evidence is “inconclusive,” that is itself an informative finding.