13  Bounds methods ✎ Draft

The methods in this chapter rest on the properties of bounded data described in Chapter 7: on a scale with a minimum and a maximum, the mean and the SD are not free to take any values independently of one another.

13.1 Mean inside range

The simplest bounds check is whether a mean lies within the range of values that its scores could take. The mean of a single item rated from 1 to 5 must lie between 1 and 5. The range of a sum score is the number of items multiplied by the lowest and highest response options, so six items rated from 1 to 5 give sum scores from 6 to 30, and their mean must lie between 6 and 30. A percentage must lie between 0 and 100. When an article reports the smallest and largest observed scores, the mean must also lie between those. And a mean exactly at the minimum or maximum is only possible if every participant had that score, in which case the SD must be zero (Section 7.2).

Working out the range requires knowing the number of items, the response options, how they were coded (for example, from 0 or from 1), and whether items were summed or averaged. Articles sometimes describe these inconsistently, so a mean outside the range shows that either the mean or the description of the measure is wrong, without saying which.

The same reasoning applies to the means of subgroups. When the subgroups together make up the whole sample, without overlap, the mean of the whole sample is a weighted average of the subgroup means, so it must lie between the smallest and the largest of them. The same is true of percentages.

Haller et al. (2022) reported the characteristics of 9,496 participants in their Table 1, both for the whole sample and for eight regions, whose sample sizes add up to 9,496. In the whole sample, 5.1% of participants were divorced, but the table reports 0.0% divorced in every one of the eight regions. Likewise, 5.5% of the whole sample lived with friends or roommates, but 0.0% in every region. A percentage for the whole sample cannot be larger than the percentages in all of the groups that make it up, so these rows of the table cannot be correct. The regression in the same article’s Table 5 includes being divorced and living with friends or roommates as predictors, with estimated coefficients, which suggests that participants in these groups did exist and that the regional zeros are the error.

The same article also describes the ranges of two of its measures inconsistently. The sum score of six prosocial behaviour items, each rated from 1 to 5, is described as ranging from 6 to 36 in the Method section, but from 6 to 30 in Table 4, which is correct. The fourteen well-being items are described as rated on a “6-point” scale from 0 to 6, which has seven points, while the range of the sum score is given as 0 to 70, which is only possible if each item was scored from 0 to 5. None of these means falls outside its range, but the inconsistencies show why the range of a measure should be worked out from its items, not taken from the article.

13.2 SD bounds

On a bounded scale, the mean and the SD are not free to vary independently of one another (Section 7.2). The scale’s minimum and maximum, the sample size, and the mean together set an upper bound on the SD, and once the data are known to be whole numbers, or to include a particular minimum and maximum, they set a lower bound too. A reported SD outside these bounds cannot be correct. Hussey et al. (2026) call this check BRIMMER, for Bounds-Related Inconsistency of Means and Errors Reported, by analogy with GRIMMER (Section 12.3). Its counterpart for the mean alone, BRIM, asks whether the reported mean can be reached at all, as in Section 13.1.

The bounds come from a series of nested constraints, each of which can only tighten them (Hussey et al., 2026):

  • The scale limits. The most spread out any set of scores can be is half at the minimum and half at the maximum. On a scale with range R, the sample SD can never exceed R / √2, which is reached by a sample of two people, one at each end.
  • The sample size. With more people, the largest possible SD falls towards R / 2. For an even sample size it is (R / 2) × √(n / (n − 1)), and for an odd one slightly less, because the people cannot be split evenly between the two ends (Popoviciu, 1935). On a 1–7 scale with 30 people, no SD can exceed 3.05, whatever the mean.
  • The mean. The further the mean is from the middle of the scale, the smaller the largest possible SD, which is √(n / (n − 1) × (max − M)(M − min)). This is the Bhatia–Davis bound of Section 7.2, which Muilwijk (1966) had already derived (Bhatia & Davis, 2000). It can be sharpened slightly, because the number of people at each end of the scale must be a whole number. On a 1–7 scale with 30 people and a mean of 2.97, the SD can be at most 2.84.
  • Integer data. Up to this point, the smallest possible SD is zero: everyone could have the same score. But when every response is a whole number, a mean with a fractional part, such as 2.97, cannot be produced by identical scores, so the SD must be above zero. This floor is usually small: 0.14 in the example.
  • Internal consistency. For a score made up of several items, a reported Cronbach’s α limits how much the items can disagree with one another, which lowers the ceiling and, for whole-number items, raises the floor (Hussey et al., 2026).

The scale limits are not the only minimum and maximum that constrain the SD. Many articles report the smallest and largest scores actually observed, in a table of descriptive statistics or in the text, and these say more than the scale limits do. The scale limits only say that no score lies outside them, whereas an observed minimum and maximum say that at least one person had each of those scores. A sample that includes both a low and a high score cannot have a very small SD, so the observed minimum and maximum raise the floor substantially, especially together with the mean and integer data. They also narrow the range of means that are possible at all, and they lower the ceiling, because the observed range is usually narrower than the scale’s. In the example above, if the article also reported that scores ranged from 1 to 6, the SD could be no smaller than 0.71 and no larger than 2.46.

When checking reported values, the bounds have to take rounding into account: a mean reported as 2.97 could have been anything from 2.965 to 2.975, and the bounds are calculated over that whole interval. Like GRIM and GRIMMER, BRIMMER is a necessary condition rather than a sufficient one. A reported SD outside the bounds is impossible, but one inside them may still not correspond to any real set of whole-number scores. How the bounds combine with the granularity checks, and a further, exact test, BRIMMEST, which settles whether any such set of scores exists, are the subject of Chapter 14.

Put the two constraints together, and the mean and the SD carry information about one another. When the scores are whole numbers and the scale has a minimum and a maximum, knowing the mean limits what the SD can be, and knowing the SD limits what the mean can be. Together, they also limit the distributions of scores that could have produced them: a large SD means that the scores must be piled up near the ends of the scale, and a small one means that they must be bunched together, and in a small sample a mean and SD can leave only one or a few sets of scores that produce them. Neither constraint is enough on its own. Continuous data on a bounded scale can take any SD between zero and the ceiling, and whole-number data without a minimum or maximum are subject only to the granularity constraints of Chapter 12, which say nothing about how large the SD can be.

The plot below shows this in the simplest case: a single item answered in whole numbers, with a small sample on a short scale, so that every set of scores that could have been observed can be listed. Each dot on the left is a combination of mean and SD that at least one set of scores produces, and the shaded band runs from the smallest to the largest possible SD at each mean, from the bounds above. The dotted curve is the uniform-spread bound from Section 7.2: above it, the scores are more spread out than if they were spread evenly across the whole scale. Click or drag in the plot to choose a combination: the choice snaps to the nearest dot, and the plot on the right shows the frequencies of a set of scores that produces it. When more than one set does, Show another steps through them.

Three things stand out. First, the dots fill the band only patchily. A mean and an SD inside the bounds need not be produced by any set of whole-number scores, which is why the bounds are a necessary condition rather than a sufficient one, and why the granularity checks of Chapter 12 add to them. Second, the dots along the top edge of the band are the most spread out samples possible at each mean: choose one, and the scores on the right sit at the two ends of the scale, with at most one person in between. The dots along the bottom edge are the least spread out, with everyone on one response or two neighbouring ones. Increase the sample size, or widen the scale, and the dots crowd together, until they are too many to tell apart. This is why, in practice, the bounds are calculated rather than every possible sample being listed. Third, the mean and SD say a good deal about the shape of the data. Choose a dot above the uniform-spread bound, and the scores that produce it are piled up towards the ends of the scale. Choose one near the floor, and the scores are bunched on neighbouring responses. With a small sample, many dots are produced by only one set of scores, so the mean and SD alone are enough to recover every score.

The next plot drops these restrictions and shows the bounds for any scale, sample size, and mean. The shaded band shows the combinations of mean and SD that are possible. Move the sliders, or click or drag anywhere in the plot, to choose a mean and an SD, and the point shows whether that combination is possible. Choose whether responses are continuous or whole numbers, and whether the score is a single item, or the sum or mean of several items, each rated on the scale. For a score made up of several items, tick Cronbach’s α to add a reported α. Tick Observed minimum and maximum to see how much a reported minimum and maximum sharpen the bounds; these are in the units of the score. The dashed grey curve is the Bhatia–Davis bound, which ignores that people come in whole numbers, and the dotted horizontal line is the largest SD that any sample of this size can have, whatever its mean. The dotted curve is the uniform-spread bound from Section 7.2. It is not a bound on what is possible, but on what is plausible: above it, the scores are more spread out than if they were spread evenly across the whole scale, which requires them to pile up at its ends. This plot shows the bounds only. It treats the mean and SD as exact, does not allow for rounding, and does not check granularity, so it does not ask whether a mean or SD could have come from whole-number data at all, as GRIM and GRIMMER do (Chapter 12). It is for building intuition about the bounds. To check values reported in an article, use the app in Section 14.3, which combines the bounds with the granularity checks.

Start with the mean at 3 on the 1–7 scale and raise the SD until the point turns red: the largest possible SD is close to the Bhatia–Davis bound, but slightly below it, because the number of people at each end of the scale has to be a whole number. On the way, the point turns amber as it crosses the uniform-spread bound, where the scores would have to pile up at the ends of the scale. Move the mean towards either end and the largest possible SD falls. With whole-number responses, the smallest possible SD is zero only when the mean is a whole number, and rises to a small peak between whole numbers. Now choose a mean of three items and tick Cronbach’s α: the higher the α, the lower the ceiling, because items that agree with one another cannot produce scores as spread out as items that disagree. Finally, tick Observed minimum and maximum: with scores known to include both the minimum and the maximum given, the band lifts well off the floor, the ceiling falls, and means near the ends of the scale become impossible.

To check values reported in an article, use the app in Section 14.3, which combines these bounds with the granularity checks of Chapter 12 and allows for rounding.

13.3 Estimates inside confidence intervals

Most estimates in quantitative research are reported with a confidence interval: a mean difference, a regression coefficient, an odds ratio, or a correlation, followed by its lower and upper bounds. The estimate and its interval are calculated from the same data by the same analysis, so they are not free to take any values relative to one another. This gives two checks that need nothing but the three reported numbers. The first is a possibility check: is the estimate inside its interval? The second is a plausibility check: is the estimate in the middle of its interval, as it should be for many common methods, once rounding is taken into account?

13.3.1 Possibility: is the estimate inside its interval?

For almost every method used in practice, a confidence interval contains its own point estimate. An estimate outside its interval is therefore close to proof that at least one of the three numbers is wrong. The exceptions are rare and exotic, such as percentile bootstrap intervals for heavily biased statistics, and an article that used such a method would be expected to say so. More common explanations are a typo, digits transposed while copying, values copied from the wrong row or the wrong model, or bounds and estimates entered in the wrong order. An estimate that equals one of its bounds is technically inside the interval, but for most methods it is no more plausible than one outside it.

Georgescu & Wren (2018) automatically extracted 244,750 ratios (odds ratios, risk ratios, and hazard ratios) and their 95% confidence intervals from the abstracts of articles indexed in MEDLINE. Between 0.4% and 0.6% of the ratios, depending on the type, were outside their own intervals. A reanalysis of their data for this book, using the 151,461 ratios whose values could be checked with the rounding of each value taken into account, found that 0.43% were outside their intervals, and that 0.8% of the abstracts contained at least one such ratio. These are small percentages, but they come from abstracts, which usually contain an article’s main results, and almost every one of them points to an error, in the article or, occasionally, in the automated extraction.

13.3.2 Plausibility: is the estimate in the middle of its interval?

Many confidence intervals are symmetric around their estimate. The usual interval for a mean, a mean difference, or a coefficient in a linear regression is the estimate plus or minus a multiple of its standard error, so the estimate lies exactly halfway between the bounds. Intervals for ratios such as odds ratios, risk ratios, and hazard ratios are usually calculated in the same way on the log scale and then exponentiated, so the estimate lies halfway between the bounds on the log scale: it is the geometric mean of the two bounds, √(lower × upper), rather than their arithmetic mean. This was the basis of the recalculations by Georgescu & Wren (2018).

Rounding means that the reported estimate need not be exactly in the middle of the reported bounds. Each of the three numbers was rounded separately, so the check asks whether any combination of unrounded values that would round to the three reported numbers places the estimate exactly in the middle. For example, an odds ratio reported as 8.4 [3.7, 19.2] passes: the geometric mean of the bounds is 8.43, which rounds to 8.4. An odds ratio of 1.52 [1.18, 1.93] does not: whatever the unrounded bounds were, their geometric mean lies between 1.504 and 1.514, and no unrounded estimate between 1.515 and 1.525 can equal it. The difference is small, but it is larger than rounding can produce.

Unlike an estimate outside its interval, an estimate that is off-centre is not proof of an error, because many legitimate methods produce intervals that are not symmetric around their estimate on any simple scale. Examples include profile likelihood intervals, which R uses by default for the coefficients of logistic and other generalised linear models; exact intervals, such as Clopper–Pearson intervals for proportions and conditional intervals for odds ratios from small samples; Wilson intervals for proportions; intervals for correlations calculated through Fisher’s z transformation, which are symmetric on the z scale but not on the scale of the correlation; intervals for Standardized Mean Differences based on the noncentral t distribution; bootstrap intervals, which can be asymmetric even when the model itself assumes normally distributed errors, because they follow the shape of the bootstrap distribution rather than a formula; and Bayesian credible intervals from skewed posterior distributions. Whether an off-centre estimate is a concern therefore depends on the metric and on the method used to calculate its interval. An off-centre coefficient from an ordinary least squares regression with standard intervals is a strong sign of an error. An off-centre odds ratio from a profile likelihood interval is expected.

In the reanalysis of the data of Georgescu & Wren (2018), 92.7% of the ratios were centred on the log scale given rounding, 6.8% were inside their intervals but off-centre, and 0.43% were outside their intervals. The histograms below show where each estimate fell, as a proportion of the width of its interval on the log scale: 0% is the lower bound, 50% the middle, and 100% the upper bound. Most off-centre estimates are only slightly off-centre, and many of them are probably the result of legitimate asymmetric methods, which these abstracts rarely describe. About 2.1% of all ratios were both off-centre given rounding and outside the middle fifth of their interval (40% to 60% of its width), which is much harder to explain by method alone.

13.3.3 Seeing the problem

Problems of this kind are much easier to see in a plot than in a table. Reading down a column of numbers such as −0.05 [−0.50, −0.11], it is easy to miss that the estimate is not between its bounds, and almost impossible to judge by eye whether an estimate is in the middle of its interval. Plotted, an estimate outside its interval, or one that sits far from the middle of it, stands out immediately.

Haller et al. (2022) reported a multiple regression predicting prosocial behaviour from demographic and psychological variables, with standardised coefficients and 95% confidence intervals in their Table 5. As a comment on PubPeer pointed out, the coefficient for being divorced, β = −0.05, is outside its interval of [−0.50, −0.11], which is not possible. The article describes an ordinary multiple regression, for which symmetric intervals are expected, but the coefficient for being widowed, β = −0.29 [−0.34, −0.01], is far from the middle of its interval, and the coefficient for high perceived stress, β = 0.29 [0.29, 0.04], equals one of its bounds, which are also in the wrong order. A coefficient reported in the text, β = 0.15 [0.12, 0.22] for Western Europe, which the same comment quoted for a different reason, is also off-centre. Each of these is easy to miss in the table. Load the table into the app below with the Haller et al. (2022), Table 5 button to see how they look in a plot. The Odds ratios from MEDLINE abstracts button loads seven odds ratios from the abstracts analysed by Georgescu & Wren (2018), as extracted, to be checked on the log scale.

The app below plots any number of estimates and their intervals. Enter a label, the estimate, and the lower and upper bounds in each row, or paste a block of cells from a spreadsheet into the cell where the block should start. Enter the values exactly as reported, including trailing zeros. The number of decimal places in each value determines how much rounding is allowed for, and so how wide the region that counts as the middle of the interval is: a bound reported as 0.30 could have been anything from 0.295 to 0.305, but one entered as 0.3 could have been anything from 0.25 to 0.35. Spreadsheets often drop trailing zeros, so if you paste values from one, check that they still have the decimal places they were reported with. Choose whether the intervals should be symmetric on the original scale, as for means, mean differences, and regression coefficients, or on the log scale, as for odds ratios, risk ratios, and hazard ratios. Each estimate is coloured green if it is in the middle of its interval given rounding, orange if it is inside its interval but not in the middle, and red if it is outside its interval. The grey band on each interval shows the range of values that would be in the middle of the interval, given rounding. The Standardise interval widths button rescales every interval to run from 0% (the lower bound) to 100% (the upper bound), on the scale on which it should be symmetric, and marks the middle at 50%. Each estimate is then shown at its position within its own interval, and the grey band shows the region around 50% that counts as the middle, given rounding. The short coloured bar around each estimate shows the values it could have had before it was rounded, and an estimate counts as in the middle if this bar touches the grey band. This makes intervals of very different widths easy to compare, and shows how much the rounding of the reported values matters: values reported with fewer decimal places, relative to the width of their interval, have a wider band. Clicking the button again shows the reported values.

Some statistical software prints the lower bound before the estimate, as in “lower, estimate, upper”, and a value copied from such output into the wrong column can produce an estimate outside its interval. The Swap estimates and lower bounds button swaps the two columns in every row, so that you can see whether the values make sense when read in the other order. Clicking it again swaps them back.

Swapping the columns does not let the authors off the hook, even when it makes every estimate sit neatly in the middle of its interval. Readers, reviewers, and meta-analysts usually take reported numbers at face value. A meta-analyst who extracts the estimate and calculates its standard error from the width of the interval, as is common, will use the wrong estimate, the wrong standard error, or both, and nothing in the meta-analysis will reveal the error. An error that would be obvious in a plot of one study can therefore distort the pooled results of many.

13.4 Coefficient bounds

Many coefficients have bounds that follow from how they are defined, whatever was measured. A correlation must lie between −1 and 1, and a proportion or an R2 between 0 and 1. Reliability, defined as the proportion of the variance in scores that is true-score variance, also lies between 0 and 1, but the coefficients used to estimate it are bounded less tightly. Cronbach’s α cannot exceed 1, but it has no lower bound: it is negative whenever the items are, on average, negatively correlated with one another, as happens when reverse-keyed items have not been reverse-scored, and two items that correlate −.90 give an α of −18. A negative α is therefore possible, but it shows that the items do not measure one thing in the way they were scored, and so it is a reason to look closely. A reported value outside a coefficient’s bounds cannot be correct. Table 13.1 lists the bounds of some statistics that are common in psychology and medicine.

Table 13.1: Bounds of some common statistics.
Statistic Bounds Notes
Pearson’s r, Spearman’s ρ, Kendall’s τ, point-biserial and phi correlations −1 to 1 For phi and point-biserial correlations, the attainable maximum is often smaller, depending on the proportions in each category.
Proportions, probabilities, and p-values 0 to 1 (0% to 100%) Includes prevalence, response rates, and proportions of variance.
Sensitivity, specificity, positive and negative predictive values, accuracy, precision, recall, and F1 score 0 to 1 All are proportions, and the F1 score is the harmonic mean of precision and recall, so it lies between them.
Area under the ROC curve (AUC, c-statistic) 0 to 1 0.5 is chance. Values below 0.5 mean that the predictor works in the opposite direction to the one assumed.
R2, η2, partial η2, Cramér’s V 0 to 1
Adjusted R2 −p / (n − p − 1) to 1, with p predictors Can be slightly negative when R2 is close to zero: at least −0.03 with 3 predictors and 100 people. It is only substantially negative when there are many predictors for the sample size.
ω2 (one-way ANOVA) −dfeffect / (N − dfeffect) to 1 Can be slightly negative when the effect is close to zero: at least −0.01 for two groups and 100 people. Often reported as 0 when negative.
ε2 (one-way ANOVA) −dfeffect / dferror to 1 As for ω2: at least −0.01 for two groups of 50.
Standardised regression coefficients (β) −1 to 1 with one predictor With several correlated predictors, values beyond ±1 are possible, but rare, and worth a closer look.
Standardised factor loadings −1 to 1 A value beyond ±1, known as a Heywood case, indicates an improper solution.
Cronbach’s α Up to 1, with no lower bound Negative when the items are on average negatively correlated, usually because reverse-keyed items were not reverse-scored. A slightly negative α is possible but a sign of a problem with the scale or its scoring. A value far below 0, such as −5, needs items that are strongly negatively correlated, which is implausible for a published scale unless the article explains it.
McDonald’s ω 0 to 1
Single-measure intraclass correlation (one-way, or two-way consistency) −1 / (k − 1) to 1, with k raters or measurements per subject −1 to 1 with two raters, and closer to 0 with more. Estimates are negative when the variance between subjects is close to zero, and are then usually only slightly negative. The two-way agreement form can fall slightly further below in very small samples.
Average-measure intraclass correlation Up to 1, with no lower bound The consistency form is identical to Cronbach’s α, and is negative under the same circumstances.
Cohen’s κ −1 to 1 The attainable maximum is often below 1, depending on the marginal proportions.
Odds ratio, risk ratio, hazard ratio Above 0 1 means no difference. Ratios cannot be zero or negative, so neither can the bounds of their intervals, although a small lower bound can be rounded to 0.00.
Risk difference −1 to 1
Number needed to treat (NNT) 1 or more in size An NNT between −1 and 1 would imply a risk difference greater than 1 in size.
Variances, SDs, and standard errors 0 or more For a single mean, the standard error, SD / √N, cannot be larger than the SD.
I2 0% to 100%
τ2, χ2, F, RMSEA, SRMR 0 or more
CFI 0 to 1 The TLI is not bounded in the same way, and can fall below 0 or rise above 1.
Cohen’s d and other Standardized Mean Differences No bounds by definition On a bounded scale, the bounds of the scale limit how large a Standardized Mean Difference can be (Chapter 7).

Some of the statistics in Table 13.1 are bias-corrected or variance-based estimates that can legitimately fall slightly below zero when the quantity they estimate is close to zero, such as adjusted R2, ω2, and the intraclass correlation. Being able to fall slightly below zero does not mean that they have no bounds. Most of them have a lower bound that follows from the sample size or design, and it is usually close to zero. An R2 of −4.1 is impossible, because R2 cannot be negative at all, and an adjusted R2 that negative would need an R2 of zero and almost as many predictors as people. A single-measure intraclass correlation of −9.1 is also impossible in practice: the one-way and consistency forms cannot fall below −1 with any number of raters, and the agreement form can only do so in samples of a few subjects. Only a few statistics, such as Cronbach’s α and the average-measure intraclass correlation, have no lower bound at all, and even for these, a value far below zero is implausible without a specific explanation.

Values this obviously wrong still reach print. Hussein et al. (2022) compared machine learning models for classifying lung diseases from chest X-rays, and reported each model’s accuracy, precision, recall, and F1 score. The F1 score is the harmonic mean of precision and recall, 2 × precision × recall / (precision + recall), so it must lie between them, and like them between 0 and 1. As a comment on PubPeer pointed out, every F1 score in the article’s Tables 2 to 4 is above 1, ranging from 1.40 to 3.64. They are not percentages, because the precision and recall in the same rows are reported as proportions. Almost all of them are four times the recall in the same row: one model was reported with a precision of .6900, a recall of .6891, and an F1 score of 2.7563, whereas the F1 score implied by its precision and recall is .6895. Whatever the cause, none of the F1 scores can be correct. Later comments on the same article raise further questions about the other metrics in these tables.

Errors like these are not rare. Bogdan (2025) extracted the statistics reported in thousands of published psychology articles. The resulting dataset contains 110,016 reported correlations, and 0.8% of them lie outside −1 to 1, so they cannot be correct.

Values that only break a bound once they are combined with other reported numbers are probably more common, because no one checks the combination. The next section describes one such check.

13.4.1 Disattenuated correlations

Measurement error weakens correlations. If two measures are not perfectly reliable, the correlation between them is smaller in size than the correlation between the constructs they measure, a bias known as attenuation. Spearman (1904) showed how to correct for it: divide the observed correlation by the square root of the product of the two measures’ reliabilities,

\[ r_{xy}^{\text{true}} = \frac{r_{xy}}{\sqrt{r_{xx} \, r_{yy}}}, \]

where rxy is the observed correlation and rxx and ryy are the reliabilities of the two measures, such as Cronbach’s α or a test–retest correlation. The disattenuated correlation is an estimate of the correlation between the constructs, and like any correlation it must lie between −1 and 1.

This gives a check that uses three numbers which articles often report together. Rearranging the formula, a correlation can only be as large as the square root of the product of the reliabilities: two measures with reliabilities of .70 can correlate at most .70, however closely related the constructs are. A reported correlation of .80 between them implies a disattenuated correlation of 1.14, which is impossible. Either the correlation, at least one of the reliabilities, or both are wrong, or the reliabilities were estimated in a different sample from the correlation.

Whether a disattenuated correlation outside −1 to 1 shows that the reported numbers are impossible, or only that they are implausible, depends on where the reliabilities come from. When the correlation and both reliabilities were estimated in the same sample, as when an article reports all three, they describe the same data, and the check is a possibility check: the numbers cannot all be correct, subject to the caveats about sampling error and the type of reliability below. When the reliabilities come from other samples, such as a test manual, earlier studies, or a meta-analysis, the check is a plausibility check. The reliabilities then describe what is typical for the measures, and the measures may have been more or less reliable in the sample at hand, so a disattenuated correlation outside −1 to 1 shows that the reported correlation is implausible given what is known about the measures, not that it is impossible. For the same reason, √(rxx ryy) is a ceiling on the correlation to be expected between two measures, not on the correlation that can be observed in any one sample, which varies around its expected value.

The plausibility check can be applied to meta-analyses as well as to single studies. Vahey et al. (2015) reported a meta-analytic correlation of r = .45 between the Implicit Relational Assessment Procedure (IRAP), a reaction-time-based measure, and clinical criterion measures. A later meta-analysis estimated the IRAP’s internal consistency at α = .27 and its test–retest reliability at .18 (Hussey & Drake, 2020). As Hussey (2025) pointed out, these estimates are incompatible with one another. Even if the criterion measures were assumed to be highly reliable (ryy = .90), the disattenuated correlation would be .91 using the IRAP’s internal consistency and 1.12 using its test–retest reliability. With a lower and arguably more plausible criterion reliability of .70, it would be 1.04 or 1.27. Either the meta-analytic correlation was too large, or the meta-analytic reliabilities were too small. A reanalysis of the original meta-analysis found serious errors that generally inflated the effect size, and a new meta-analysis calculated to show their combined impact produced an estimate half the size, r = .22 (Hussey, 2025).

Enter a reported correlation and the reliabilities of the two measures below, and choose whether the reliabilities come from the same sample as the correlation, which makes the check a possibility check, or from other samples, which makes it a plausibility check. The plot shows the reported correlation and the disattenuated correlation, joined by an arrow; the grey areas outside −1 and 1 are impossible for any correlation. You can also click or drag anywhere on the plot to set the reported correlation. The app uses the values exactly as entered and does not allow for their rounding, so it is better treated as an intuition pump than as a formal test.

Try a correlation of .80 with both reliabilities at .70: the disattenuated correlation is 1.14. Then raise either reliability until the correlation becomes possible. Negative correlations work the same way, against the lower bound of −1.

Some points to keep in mind when using this check:

  • Reliabilities from other samples are still informative. When an article does not report reliabilities, estimates from a test manual, earlier studies, or a reliability generalization meta-analysis can stand in for them, and the check becomes a plausibility check. It is often said that reliability is a property of scores in a particular sample rather than of a measure, but this is too strong. Reliability generalization meta-analyses, which pool the reliabilities reported for a measure across many samples, find that reliability varies between samples but that the measure itself also contributes to it (Vacha-Haase, 1998): some measures are reliable in most of the samples in which they are used, and others rarely are. A correlation that would be impossible given a measure’s typical reliability is not proof of an error, but it is a useful intuition pump and a fair reason to ask questions.
  • Small excesses can arise from sampling error. The correlation and the reliabilities are all estimates, and in small samples a disattenuated correlation slightly above 1 can occur by chance even when every reported number is correct. A disattenuated correlation of 1.02 in a sample of 50 is weak evidence of a problem. One of 1.30 in a sample of 500 is strong evidence.
  • Rounding is not taken into account. The app uses the values exactly as entered. A disattenuated correlation just above 1, calculated from values rounded to two decimal places, may be within rounding of a possible one, so the app is better treated as an intuition pump than as a formal test.
  • The type of reliability matters. Internal consistency coefficients such as Cronbach’s α can underestimate or overestimate reliability, depending on whether their assumptions hold, which changes the disattenuated correlation in turn. When an article reports more than one kind of reliability, check the result with each.
  • A disattenuated correlation close to 1 is itself informative. Even when it is possible, a disattenuated correlation near 1 suggests that the two measures assess the same construct, which may sit uneasily with an article’s claim that they measure distinct things.

13.5 Test yourself

Answer all the questions, then click the button below the quiz to see why each answer is right.

Means and SDs

  1. A sum score of six items, each rated from 1 to 5, can range from 6 to 30.
  2. When subgroups together make up a whole sample, the mean of the whole sample must lie between the smallest and largest subgroup means.
  3. A table reports that 5.1% of a sample were divorced, and 0.0% in every one of the regions that make up the sample. This is possible if some of the regions are small.
  4. If the mean of a single item rated from 1 to 5 is exactly 5, the SD must be 0.
  5. On a 1–7 scale with 30 people, an SD of 3.20 is:
  6. At a given mean on a bounded scale, the largest possible SD is reached when:
  7. With whole-number responses, a mean of 2.97 can have an SD of 0.
  8. A reported minimum and maximum can make a small SD impossible, even though the same SD would be possible if only the scale limits were known.
  9. A reported Cronbach’s α can make a large SD impossible, even though the same SD would be possible without it.
  10. A reported SD that lies within the BRIMMER bounds has been shown to be possible.
  11. An SD above the uniform-spread bound is impossible.
  12. With a small sample on a short scale, a reported mean and SD can be enough to recover every individual score.

Estimates inside confidence intervals

  1. An estimate that lies outside its own 95% confidence interval is:
  2. Among ratios reported with confidence intervals in MEDLINE abstracts, roughly what share were outside their own intervals?
  3. An odds ratio should lie halfway between the lower and upper bounds of its confidence interval.
  4. An estimate that is inside its interval but not in the middle of it, after allowing for rounding, is proof of an error.
  5. Why should values be entered into the confidence interval app with their trailing zeros?
  6. If swapping each estimate with its lower bound puts every estimate in the middle of its interval, the article’s values can be used as reported.

Coefficients

  1. An article reports an F1 score of 2.76, alongside a precision of .69 and a recall of .69. The F1 score could be correct.
  2. An adjusted R2 slightly below zero can be correct.
  3. An ordinary R2 can be negative when a model fits very badly.
  4. Cronbach’s α can be negative.
  5. The standard error of a single mean can be larger than the SD.
  6. Two measures each have a reliability of .70. What is the largest correlation to expect between them?
  7. An article reports a correlation and the reliabilities of both measures, all estimated in its own sample, which give a disattenuated correlation of 1.14. This is a possibility check rather than a plausibility check.
  8. A correlation of .80 is reported between two measures whose reliabilities, taken from their test manuals, are both .70. This shows that the correlation is impossible.
  1. True. The range of a sum score is the number of items times the lowest and highest response options: 6 × 1 = 6 and 6 × 5 = 30. Work the range out from the items rather than taking it from the article, which may describe it inconsistently (Section 13.1).
  2. True. The mean of the whole sample is a weighted average of the subgroup means, so it cannot be smaller than all of them or larger than all of them. The same is true of percentages (Section 13.1).
  3. False. However small or large the regions are, a percentage for the whole sample cannot be larger than the percentages in every one of the groups that make it up. If every region has 0.0%, so must the whole sample (Section 13.1).
  4. True. A mean exactly at the maximum is only possible if every participant had the maximum score, and then there is no spread at all (Section 13.1).
  5. Impossible, whatever the mean. With 30 people on a 1–7 scale, no SD can exceed 3.05, which is reached with half the people at each end of the scale (Section 13.2).
  6. As many scores as possible are at the two ends of the scale. The most spread out samples have their scores at the two ends, with at most one person in between. Moving the mean away from the middle of the scale lowers this ceiling (Section 13.2).
  7. False. An SD of 0 means that everyone had the same score, and identical whole numbers cannot have a mean with a fractional part, such as 2.97. Whole-number data raise the floor above zero at every mean that is not a whole number (Section 13.2).
  8. True. A sample that includes both the observed minimum and the observed maximum cannot have a very small SD. On a 1–7 scale with 30 people and a mean of 2.97, reporting a minimum of 1 and a maximum of 6 raises the floor from 0.14 to 0.71 (Section 13.2).
  9. True. A reported α limits how much the items can disagree with one another, which lowers the ceiling on the SD of the total or mean score (Section 13.2).
  10. False. BRIMMER is a necessary condition, not a sufficient one. A reported SD inside the bounds may still not correspond to any real set of whole-number scores; only BRIMMEST settles that (Section 13.2, Chapter 14).
  11. False. The uniform-spread bound is a bound on what is plausible, not on what is possible. Above it, the scores are more spread out than if they were spread evenly across the whole scale, which requires them to pile up at its ends. That is plausible for some measures, but not most (Section 13.2).
  12. True. In a small sample on a short scale, many combinations of mean and SD are produced by only one set of whole-number scores, so the mean and SD alone reveal every score (Section 13.2).
  13. Almost always an error. For almost every method used in practice, a confidence interval contains its own estimate. The exceptions are rare, and an article that used one would be expected to say so (Section 13.3.1).
  14. Less than 1%. Between 0.4% and 0.6% of ratios, depending on the type, were outside their intervals, and 0.43% once rounding was taken into account. The share is small, but almost every one points to an error (Section 13.3.1).
  15. False. Intervals for ratios are usually calculated on the log scale, so the estimate lies halfway between the bounds on that scale: it is the geometric mean of the bounds, not their arithmetic mean (Section 13.3.2).
  16. False. Many legitimate methods produce asymmetric intervals, such as profile likelihood, exact, Wilson, and bootstrap intervals. Whether an off-centre estimate is a concern depends on the metric and on the method the article used (Section 13.3.2).
  17. The number of decimal places sets how much rounding is allowed for. A bound reported as 0.30 could have been anything from 0.295 to 0.305, but one entered as 0.3 could have been anything from 0.25 to 0.35 (Section 13.3).
  18. False. Even if the columns were simply swapped, readers and meta-analysts usually take reported values at face value, and a meta-analyst who uses the wrong estimate or calculates the wrong standard error will distort the pooled results (Section 13.3).
  19. False. The F1 score is the harmonic mean of precision and recall, so it must lie between them, and like them between 0 and 1. With a precision and recall of .69, the F1 score is .69 (Section 13.4).
  20. True. Adjusted R2 is bias-corrected, and can fall slightly below zero when R2 is close to zero. Its lower bound depends on the number of predictors and the sample size, and is usually close to zero (Section 13.4).
  21. False. R2 is a proportion of variance, so it lies between 0 and 1. A reported R2 below zero cannot be correct (Section 13.4).
  22. True. Cronbach’s α has no lower bound, and is negative when the items are on average negatively correlated, usually because reverse-keyed items were not reverse-scored. A negative α is possible, but a sign of a problem with the scale or its scoring (Section 13.4).
  23. False. The standard error of a single mean is SD / √N, so it cannot be larger than the SD (Section 13.4).
  24. .70. The largest correlation to expect is the square root of the product of the reliabilities: √(.70 × .70) = .70, however closely related the constructs are (Section 13.4.1).
  25. True. When the correlation and both reliabilities describe the same data, they cannot all be correct if the disattenuated correlation is outside −1 to 1, subject to sampling error, rounding, and the type of reliability used (Section 13.4.1).
  26. False. The disattenuated correlation would be 1.14, but because the reliabilities come from other samples, this is a plausibility check, not a possibility check: the measures may have been more reliable in this sample. It is a good reason to ask questions, but not proof of an error (Section 13.4.1).