6  Standard Deviations vs. Standards Errors ✎ Draft

The standard deviation (SD) and the standard error (SE) are both reported after a “±” or in parentheses after a mean, they have similar names, and they are calculated from the same data. They nevertheless answer different questions, and they can differ from each other by an order of magnitude. Mistaking one for the other is one of the most common errors in the reporting of research and in the extraction of results for meta-analysis, and it is one of the main reasons that implausibly large standardised effect sizes appear in the literature.

This chapter explains the difference between the two, shows what happens to Cohen’s d (Section 5.2) when a standard error is mistaken for a standard deviation, summarises how often this happens, and describes how to check for it.

6.1 What each one describes

The standard deviation describes the participants. It says how much individual scores vary around the mean of the sample, in the units of the scale. If one study recruits 20 people from a population and another recruits 2,000 people from the same population, the two studies should find roughly the same SD, because the SD is a property of how much people differ from one another, not of how many of them were measured.

The standard error describes the estimate. Every sample gives a slightly different mean (see the dance of the p-values, Section 4.4.0.1), and the standard error of the mean says how much the means of samples of a given size vary from sample to sample. It is the standard deviation of the sampling distribution of the mean, and it is calculated from the SD and the sample size:

\[ SE = \frac{SD}{\sqrt{N}} \]

Unlike the SD, the SE depends strongly on the sample size: quadrupling the sample halves the standard error. The study with 2,000 participants has the same SD as the study with 20, but a standard error that is ten times smaller.

The plot below shows both. The top panel shows the distribution of individual scores, whose spread is the SD. The bottom panel shows the distribution of the means of samples of size N drawn from that population, whose spread is the SE. Change the sample size and notice that the top panel does not change at all, while the bottom panel narrows as N increases. Then change the SD and notice that both change.

Two things follow from this. First, the two statistics are used for different purposes. The SD is a descriptive statistic: it is what a reader needs in order to know how variable the sample was, and it is what Cohen’s d divides by. The SE is an inferential statistic: it is what confidence intervals and significance tests are built from. Reporting a standard error where a description of the sample is called for is an error of reporting, even when the number is labelled correctly (Altman & Bland, 2005).

Second, the two are never close in size except in very small samples. With 25 participants the SD is 5 times the SE; with 100 participants it is 10 times; with 400 it is 20 times. This is what makes confusing them so consequential.

6.1.1 Converting between them

Because the two differ only by a factor of \(\sqrt{N}\), each can be recovered from the other when the sample size is known. A 95% confidence interval for a mean can also be converted, because its half-width is the standard error multiplied by a critical value from the t distribution.

# standard error from standard deviation
se_from_sd <- function(sd, n) sd / sqrt(n)

# standard deviation from standard error
sd_from_se <- function(se, n) se * sqrt(n)

# standard deviation from the 95% confidence interval of a mean
sd_from_ci <- function(lower, upper, n) {
  se <- (upper - lower) / (2 * qt(.975, df = n - 1))
  se * sqrt(n)
}

se_from_sd(sd = 10, n = 25)
[1] 2
sd_from_se(se = 2, n = 25)
[1] 10
sd_from_ci(lower = 45.87, upper = 54.13, n = 25)
[1] 10.00534

In trustworthiness assessment these conversions are most useful in reverse: when it is unclear which statistic an article has reported, calculate what the SD would be under each interpretation and ask which of the two is plausible for that scale and that population.

6.2 What happens when they are confused

Cohen’s d is the difference between two means divided by the standard deviation. If the number that is divided by is in fact a standard error, the denominator is too small by a factor of \(\sqrt{n}\), where n is the sample size of each group, and the effect size is too large by the same factor:

\[ d_{confused} = \frac{M_1 - M_2}{SD / \sqrt{n}} = d \times \sqrt{n} \]

The error is therefore worse in larger studies. A true effect of d = 0.3 becomes 1.34 with 20 participants per group, 2.12 with 50 per group, and 3.00 with 100 per group. This is the opposite of what we usually expect from large studies, which are normally the ones whose estimates we trust most.

The plot below shows this. As in Section 8.2, it simulates 20,000 two-group studies at each sample size, each with a true standardised mean difference drawn from a normal distribution with mean μ and standard deviation τ. For each study it then calculates the effect size twice: once correctly, and once with the standard error of each group mistaken for its standard deviation. The lines show the median, and the bands show the 95% prediction intervals: the ranges within which 95% of studies’ observed effect sizes fall.

Median (line) and 95% prediction interval (band) of an observed d, when it is calculated correctly and when the SE is mistaken for the SD. The dashed blue line shows the mean true effect μ.

Several things are worth noticing:

  • The correctly calculated effect sizes converge on the true effect as the sample size increases. The confused ones move further away from it.
  • The confused effect sizes are not slightly inflated versions of the correct ones. With the default settings, the median confused effect size is about 2 at a total sample size of 90 and nearly 5 at a total sample size of 500, when the true effect is 0.3.
  • Even a true effect of zero is affected. Set μ to 0: the confused effect sizes are still centred on zero, but they are spread over a range many times wider than the correct ones, so that large effects in both directions become common.
  • Small true effects are turned into effects that would be among the largest ever observed in most fields. An observed d of 2 or 3 is exactly what this error produces from an unremarkable true effect in a study of ordinary size.

The opposite error also occurs: a standard deviation is mistaken for a standard error, and is multiplied by \(\sqrt{n}\) on the assumption that it needs to be converted. This shrinks the effect size by a factor of \(\sqrt{n}\). It is harder to notice, because implausibly small effects attract less attention than implausibly large ones.

6.3 Prevalence of SE/SD misuse in primary research

The error starts in primary research articles. Many articles report the standard error of the mean where a standard deviation is called for, that is, to describe the variability of the sample, and some do not say which of the two they are reporting. Either creates the conditions for the number to be misread later, by readers and by meta-analysts.

64%

of articles in three leading cardiovascular journals contained at least one instance of incorrect use of the standard error of the mean (Wullschleger et al., 2014).

Audits of the published literature in several fields of medicine have found the following:

  • Altman & Bland (2005) set out the distinction between the two statistics in a short and widely cited tutorial. They note that in one sample of articles, 14% failed even to identify which of the two was being reported.
  • Nagele (2003) found that 23% of articles in four anaesthesia journals inappropriately used the standard error of the mean to describe the variability of the sample.
  • Ko et al. (2014) found that 13.6% of articles in four obstetrics and gynaecology journals did so.
  • Wullschleger et al. (2014) found that 64% of articles in three leading cardiovascular journals contained at least one instance of incorrect use of the standard error of the mean.
  • Barde & Barde (2012) note one reason for the practice: authors sometimes prefer the standard error because it is smaller, which makes the data appear less variable.

Taken together, these audits suggest that somewhere between about 14% and 64% of primary research articles, depending on the field and on how the error is defined, report standard errors where standard deviations are appropriate or do not make clear which they are reporting.

6.4 Downstream consequences for meta-analysis

Meta-analyses of standardised mean differences are calculated from the means, standard deviations, and sample sizes extracted from each primary study. When a value that is extracted as a standard deviation is in fact a standard error, that study’s effect size is inflated by \(\sqrt{n}\), and the error is carried into the pooled estimate. Because meta-analyses are highly cited and inform guidelines and practice, the consequences are larger than those of the same error in any single study.

60%

of all effect sizes greater than 3.0 in highly cited strength and conditioning meta-analyses were attributable to a standard error being mistaken for a standard deviation (Kadlec et al., 2023).

  • Kadlec et al. (2023) audited highly cited meta-analyses in strength and conditioning research. 85% contained at least one statistical error, and 45% contained effect sizes that had been inflated by a standard error being treated as a standard deviation. 60% of all effect sizes above 3.0 were attributable to this specific error.
  • Sandercock (2024) re-analysed a highly cited meta-analysis in sports medicine and found standard error/standard deviation mix-ups in 9 of its 16 effect sizes. After correction, the pooled standardised mean difference dropped from 1.46 to 0.54, and one comparison was no longer statistically significant.
  • Kanellopoulou et al. (2025) identify the confusion of standard errors with standard deviations as one of the most common statistical errors in systematic reviews.
  • Gallardo-Gómez et al. (2024) provide a tutorial on calculating standardised mean differences for meta-analysis, including guidance on distinguishing standard errors from standard deviations during data extraction.

One feature of the error makes it particularly damaging in a meta-analysis: an affected study is not given much less weight. The sampling variance of a standardised mean difference is calculated mainly from the sample sizes, with only a small contribution from the effect size itself, so a study whose effect size has been inflated four- or tenfold still receives close to the weight it would have had anyway.

The plot below simulates a single random-effects meta-analysis of two-group studies. Set the number of studies, the sample size of each group, the true effect, and the proportion of studies in which the standard error was mistaken for the standard deviation. Every study has the same true effect. The forest plot shows each study’s effect size and 95% confidence interval as a meta-analyst would have extracted them, followed by two pooled estimates: one from the studies as extracted, and one from the same studies with the errors corrected. Click the button to simulate another meta-analysis with the same settings.

Studies in which the SE was mistaken for the SD are shown in orange, and the others in grey. The dashed blue line shows the true effect. The pooled estimates are from a random-effects model using the DerSimonian–Laird estimator.

Simulate several meta-analyses and notice the following:

  • With the default settings, only 3 of the 20 studies have the error. Across many simulated meta-analyses, this is enough to make the pooled estimate about 60% too large on average (0.81 instead of 0.50).
  • The confidence interval of the pooled estimate is nearly three times as wide as it would be with the errors corrected, but not wide enough: it contains the true effect in only about half of the simulated meta-analyses.
  • Increasing the sample size per group makes things worse, not better, because the inflation of each affected study grows with its sample size. With 100 participants per group, the same three studies make the pooled estimate more than twice the true effect on average.
  • Increase the proportion of studies with the error to 25%, and the pooled estimate is on average more than twice the true effect; at 50%, as Sandercock (2024) found in one published meta-analysis, it is more than three times the true effect.
  • The affected studies have confidence intervals that are only somewhat wider than those of the other studies. Nothing about their precision marks them out as unreliable.
  • With the proportion set to zero, the two pooled estimates are identical. The difference between them is entirely due to the error.

These consequences are examined further in a simulation study by Hussey (2026, in prep), which varies the proportion of affected studies, the sample sizes, the number of studies, and the true effect, and evaluates rules for excluding implausibly large effect sizes before pooling. Its code and results are available at github.com/ianhussey/smd-sd-se-confusion.

6.5 Checking for SE/SD confusion in trustworthiness assessment

The error is detectable more often than it is detected, because it leaves numbers that are implausible in several ways at once. When assessing an article or a meta-analysis:

  1. Check what is labelled. Does every table and figure say whether the values after “±”, in parentheses, or shown as error bars are standard deviations, standard errors, or confidence intervals? Unlabelled values should not be assumed to be standard deviations.
  2. Ask whether the reported SDs are plausible for the scale. Compare them with the range of the scale, with the SDs reported for the same measure in other studies of similar populations, and with each other across the tables of the same article. An SD that is a small fraction of what other studies report is more likely to be a standard error than a sign of an unusually homogeneous sample (see also Section 5.4).
  3. Try the conversion. Multiply the suspicious value by \(\sqrt{n}\). If the result is a typical SD for that measure, the reported value was probably a standard error.
  4. Look for SDs that shrink as groups get larger. Standard deviations should be similar across groups and subgroups of different sizes. Values that are systematically smaller in larger groups behave like standard errors.
  5. Check whether the effect size is plausible. Standardised mean differences above about 2 or 3 are rare when calculated correctly (Section 8.2) and may not even be possible on a bounded scale (Section 8.1). Kadlec et al. (2023) traced most effect sizes above 3 in the meta-analyses they audited to this error.
  6. Recalculate. Check that the reported means and SDs are consistent with the reported test statistics, p-values, and confidence intervals (see Chapter 10). A t-test calculated by the original authors from the raw data will not agree with an effect size calculated from standard errors that have been mistaken for standard deviations.
  7. In a meta-analysis, inspect the forest plot. Look for studies whose effect sizes are far larger than the rest without having wider confidence intervals, particularly among the larger studies, and trace their extracted values back to the primary articles.

One consequence of the fact that the inflation grows with sample size is that this error does not look like the small-study effects that meta-analysts are trained to look for. Funnel plots and related methods are designed to detect the pattern in which small studies show the largest effects. Here it is the larger studies that are the most distorted.

6.6 Test yourself

An article reports that anxiety scores on a 0–40 scale were lower in the intervention group (M = 14.2 ± 0.9, n = 64) than in the control group (M = 17.1 ± 1.0, n = 64), and does not say what the values after ± are. A meta-analysis that includes the study reports its effect size as d = −3.05. Mark each statement as true or false.

  1. If the values after ± are standard deviations, the meta-analysis has calculated d correctly.
  2. If the values after ± are standard errors, the SDs are about 7 to 8, and d is about −0.38.
  3. Standard deviations of about 1 on a 0–40 scale would mean that nearly everyone in each group scored within a few points of each other.
  4. The standard error would be the appropriate statistic for describing how variable the participants’ scores were.
  5. If this study had recruited 256 participants per group and the same error had been made, the effect size in the meta-analysis would be less inflated.
  6. Because the study is one of the larger ones in the meta-analysis, the error will have little influence on the pooled estimate.