Small Samples in Hypnosis Research: How Much Can We Conclude?

“Out of how many?” is the question I ask before any hypnosis headline is allowed to stand, and it’s the question this whole post is about. The old me asked it too, but only as a punchline, and only about studies I already disliked. The honest version asks it of the studies I like, answers it in one line, and then asks the harder follow-up: given that number, how much can this result actually tell us?

Plenty of hypnosis research is small. A 2014 review of hypnosis before medical procedures found 18 randomized trials ranging from 20 to 200 patients, and 14 of them had 60 or fewer [5]. That doesn’t make the field bad. It makes the field blurry, and the blur has a direction.

Why a small sample gives a blurry answer

Every trial estimates a true effect from a sample of people. With a handful of people, a few unusual responders can swing the average a long way. With hundreds, they get diluted. A confidence interval is how researchers show that blur: a range of effects that are reasonably compatible with the data.

Same effect, different sample sizesHow wide the 95% interval is around an observed d = 0.5

• 10 per group-0.39 to 1.39
• 25 per group-0.06 to 1.06
• 50 per group0.10 to 0.90
• 100 per group0.22 to 0.78

To make this concrete, here’s some arithmetic of my own, using the standard approximation for the confidence interval around a standardized effect size. Suppose four trials all observe the same moderate effect, d = 0.5, with equal groups of different sizes:

Participants per group	Approximate 95% confidence interval for d = 0.5
10	-0.39 to 1.39
25	-0.06 to 1.06
50	0.10 to 0.90
100	0.22 to 0.78

Same headline number every time. With 10 per group, the data are compatible with anything from a slight harm to a huge benefit. With 25 per group, the interval still crosses zero. Only at 50 and 100 per group does the range exclude no effect, and even then it’s wide. These are illustrations, not results from any particular study, but every small hypnosis trial you read carries an interval like the top two rows, whether or not the abstract mentions it.

This is also why conventional sample-size planning asks for more people than most hypnosis trials have. Under Cohen’s widely used conventions, detecting a medium difference between two groups (d = 0.5) with 80% power at the usual .05 level takes about 64 people per group [1]. A trial of 25 per group is not designed to reliably find an effect of that size.

Why small samples also exaggerate

Blur alone would be fine if it cut both ways equally. It doesn’t, once statistical significance and publication get involved.

When a study has low power, the only way it can reach significance is if the observed effect happens to come out large. So the small trials that “work” tend to be the ones where chance pushed the estimate up. A widely cited analysis put it plainly: low power not only reduces the chance of detecting a true effect, it also reduces the chance that a significant result reflects a true effect, and it inflates effect estimates. The same authors estimated median power in neuroscience studies at roughly 8% to 31% [2]. Gelman and Carlin gave the two failure modes names: a Type M error, where a significant estimate exaggerates the magnitude, and a Type S error, where it has the wrong sign entirely [3].

This isn’t just theory. A study of 93 medical meta-analyses covering 735 trials compared small and large trials of the same treatments. On average, effects were 48% larger in trials with fewer than 50 patients than in trials with 1,000 or more [4]. The pattern held across size groups: the smaller the trial, the bigger the reported effect.

Hypnosis research shows the same fingerprint. In a meta-analysis of 10 clinical studies on whether suggestibility predicts outcome, smaller studies tended to report larger effects [6]. With 283 participants in total, the pooled correlation was r = .24, and its 95% confidence interval ran from -0.28 to 0.75 [6]. Out of how many? Not enough to know whether the relationship is negative, modest or strong.

Where small samples show up in hypnosis research

• Procedure trials. As above, most trials before medical procedures had 60 or fewer patients [5].
• Brain imaging. A well-known imaging study of hypnotizability compared 12 high and 12 low hypnotizable adults [7]. Imaging is expensive, so this is common, and it’s why I treat single imaging findings as leads, not maps.
• Subgroups. Even a decent-sized trial becomes a small one when you split it by age, sex or suggestibility. Subgroup results inherit all the blur above.
• Long-term follow-up. Dropouts shrink the sample over time, so the later time points are usually the least certain.

What small studies can still tell you

The lazy-skeptic move, which I made often, is to wave away any trial under 100 people. That’s wrong too. Small studies earn their place in a few ways:

• Feasibility and direction. Can the intervention be delivered? Do people accept it? Do outcomes move the right way?
• Consistency across studies. If many small trials in different places point the same way, that’s more persuasive than any one of them.
• Pooling. Meta-analyses combine small trials into a larger estimate, as long as the trials are reasonably similar and publication bias is checked.
• Pre-registration. A small trial that registered its outcomes in advance and reports all of them is worth more than a bigger one that chose its outcomes afterward.

Pooling has its own honesty check. The large surgical meta-analysis of 50 trials found encouraging averages, but the 95% prediction intervals included no effect for every outcome except procedure time [8]: the next small trial in a new hospital could plausibly find nothing. And the field does have some large studies. One secondary analysis of a randomized trial of gut-directed hypnotherapy for IBS included 448 patients [9]. That’s the scale at which I start relaxing.

Claim you might see	What a single small trial can support
“Hypnosis was effective”	The direction was positive in this sample; the size is uncertain
“Large effect”	Possibly inflated; check the interval and larger trials
“No effect”	Possibly too small to detect a real, modest effect
“Works for high suggestible people”	A subgroup result, even blurrier than the main one
“Brain changes during hypnosis”	A lead worth replicating, not a mechanism

Five questions I now ask of every small trial

1. How many people were in each group, after dropouts? Not the number randomized in the abstract, the number analyzed.
2. What was the confidence interval? If it runs from roughly nothing to enormous, the honest summary is “we don’t know the size yet.”
3. Was it pre-registered, with the outcomes named in advance? Small trials with many outcomes have many chances to find one that sparkles.
4. What did the comparison group get? A small trial against a wait-list is two kinds of weak at once. See Why Comparison Groups Matter.
5. Has anyone bigger found the same thing? If the only support is other small trials from the same team, I wait.

None of these questions is a gotcha. They’re the questions I skipped for a decade, which is how I managed to be confidently wrong in both directions.

Limitations worth naming

Limitation	Consequence
Many hypnosis trials have under about 60 participants	Wide intervals; moderate effects can be missed or exaggerated
Significance filters small studies	Published small trials skew toward larger effects
Few large, independent replications	Hard to tell a true moderate effect from an inflated one
Subgroups and follow-ups shrink samples further	The most interesting secondary findings are the least certain
Abstracts rarely report intervals prominently	Readers see a clean number instead of a range

What this means in practice

A small hypnosis trial is a reason to be interested, not a reason to be sure. When you see a result, ask how many people were in each group, look for the confidence interval, and check whether larger trials or a meta-analysis with a prediction interval point the same way. A big effect in a small trial is the result most likely to shrink. A null result in a small trial may simply mean the study couldn’t see a modest effect. And a clinic page citing a single small study as proof is citing a blurry photograph as a portrait; I’d like to see the next five photographs before I believe the face.

For how missing studies make this worse, see Publication Bias in Hypnosis Research. For how pooled evidence is built, see What Is a Systematic Review?.

A note on intellectual honesty. Small samples are a constraint of funding and practicality, not a sign of bad faith, and the same problem affects every field that studies people. Reading them carefully means neither dismissing them nor overselling them. If you’re using research to make a health decision, discuss it with a qualified clinician who can weigh the whole evidence base for your situation.