Selection bias
Selection bias is the gap between the quantity a study is after in the population and the same quantity in the subset that ended up available for analysis.
Selection bias is the gap between the quantity a study is after in the population and the same quantity in the subset that ended up available for analysis.
This is a property of a sample rather than a habit of a mind, which it shares with survivorship bias, entry 8, and Berkson's paradox, entry 58, and with almost nothing else in this collection. Something decided who got into the data, and the estimate computed inside that subset answers a different question from the one asked. Hernán states the general form in his 2017 commentary as the bias that arises when the parameter of interest in a population differs from the parameter in the subset of individuals available for analysis.
The structural account came earlier. Hernán, Hernández-Díaz and Robins describe the shared causal structure as "conditioning on a common effect of 2 variables, one of which is either exposure or a cause of exposure and the other is either the outcome or a cause of the outcome". They set that against confounding, which they define as "the existence of common causes of exposure and outcome".
A common effect and a common cause are different objects and they call for opposite handling. Heckman reached a version of the problem from econometrics. He wrote in 1979 that it "may arise in practice for two reasons": self selection by the people being studied, and "sample selection decisions by analysts or data processors", which he said "operate in much the same fashion as self selection".
No replication attempt was found. That is a fact about the literature, not a verdict on the effect. The other 88.
No replication located No replication attempt was found, and none is the right thing to look for; what exists instead is a record of the bias measured in particular datasets.
No replication attempt was found for this effect. That is not evidence that it fails: the search came up empty, and an absence in the literature says nothing either way. How this index searches.
The survey sent to current customersillustration
A firm emails a satisfaction survey to everyone on its active list and reports a score of four out of five. Everyone who gave up on the product left the list first, so the people whose answers would have moved the number are the ones the sampling frame removed.
The consultation nobody young answeredillustration
A council posts a questionnaire about a park and receives nine hundred replies, most from people who attend residents' meetings on weekday evenings. The result is a genuine measurement of something. It is not the measurement of local opinion that the covering letter says it is.
The volunteers at the health fairillustration
A study recruits its participants from people who came to a free screening day. Turning up already goes with caring about one's health and with having a free morning, and both of those go with the outcome being measured, so the estimate answers a question about attenders.
Cases marked as illustrations describe nobody in particular. They are there to make the pattern recognisable, not to report that a particular thing happened.
Heckman's paper treats the bias "that results from using nonrandomly selected samples to estimate behavioral relationships as an ordinary specification error or 'omitted variables' bias", and offers "a simple consistent two stage estimator". His examples are labour-market ones. Market wages are observed for working women whose market wage exceeds their home wage at zero hours of work, and for union members who found the nonunion alternative less desirable.
The occupational literature supplies measured cases. Burns, Bodner, Jammer, Collins and Swaen calculated standardised mortality ratios for 114,683 US chemical industry employees who worked at least three days between 1960 and 2005. All-cause mortality came out at 79, 95% CI 78 to 80, against the general population, with 70 for non-malignant respiratory disease, 95% CI 67 to 73.
Employed people are not a random draw of the living, and the paper opens by noting that occupational studies typically observe a 20% deficit in overall mortality. Chen and Seaton examined 85 occupational cohorts of workers exposed to organic solvents. Their abstract gives a weighted all-cause standardised mortality ratio of 89.5, 95% CI 89.0 to 90.2, and attributes that figure to their own earlier work rather than to this paper.
What this paper adds is a regression, which finds that the higher the rates of loss to follow up, the greater the effect, p = 0.04. Psychology's version is about who volunteers. Henrich, Heine and Norenzayan, reporting an analysis by Arnett of the top journals in six subdisciplines from 2003 to 2007, write that "68% of subjects came from the United States, and a full 96% of subjects were from Western industrialized countries", and that this means "96% of psychological samples come from countries with only 12% of the world's population".
In the Journal of Personality and Social Psychology, 67% of the American samples and 80% of the samples from other countries were composed solely of undergraduates in psychology courses. Their summary is that "a randomly selected American undergraduate is more than 4,000 times more likely to be a research participant than is a randomly selected person from outside of the West".
Survey research has the best-documented failures because the truth arrives on election night. AAPOR's task force on the 2020 US general election found the average error on the margin, among polls in the final two weeks, was 4.5 points for national polls and 5.1 points for state-level presidential polls, with the signed error too favourable to Biden by 3.9 and 4.3 points.
The task force ruled out, as primary sources of the error, late deciders, failure to weight by education, the demographic composition of the electorate, differential turnout among Democratic and Republican respondents, and reluctance to name Trump to an interviewer. What it could not rule out was who declined to answer at all: "At least some of the polling error in 2020 was caused by unit nonresponse." The 2024 task force repeated the exercise and found smaller errors.
Across 611 polls fielded in the final two weeks, the average absolute error on the two-party margin was 3.3 points, against 5.3 in 2020 and 5.2 in 2016. National presidential polls missed by 2.6 points and state-level presidential polls by 3.0. That report rebuilt the historical dataset, so its own figures for earlier years differ from the ones the 2020 task force published.
15 primary research3 commentary and critique
"Sample Selection Bias as a Specification Error", Econometrica 47(1), 153 to 161. The term was already in circulation. Heckman's own reference list carries his 1976 paper on the common structure of truncation and sample selection models, Gronau's 1974 "Wage Comparisons: A Selectivity Bias" and Lewis's reply of the same year, and his footnote 2 credits Gronau and Lewis with a grouped data version of the estimation method.
What 1979 established is the treatment of the problem as an ordinary specification error with a usable correction. The structural definition that epidemiology now works from arrived 25 years later, with Hernán, Hernández-Díaz and Robins in 2004. Berkson's 1946 hospital-admission case, entry 58 here, is an earlier documented species.
Selection bias is not an experimental effect. There is no protocol to re-run, no effect size that travels between labs and no confidence interval that would settle anything. Given a selection process connected to what is being estimated, the selected sample is distorted, and that follows from the structure rather than being tested. Every number above is a measurement of one dataset.
What is genuinely contested is the scope of the structural account, and the argument has moved twice since 2004. Hernán published a commentary in 2017 arguing that "selection bias may arise when conditioning on a variable C that is not a collider". He read that off Greenland's 1977 example, in which the causal risk ratio was 1.69 in the whole population and 2.05 among the uncensored, with about 28% of people censored, and censoring associated with the outcome at a risk ratio of 1.36 but with the exposure at 1.02.
Lu, Gonsalves and Westreich pushed the other way in 2024. They argue that including a collider in a regression should not count as selection bias, because "there is no selection" and the sample analysed does not change, and that the case belongs under overadjustment bias instead. Lu, Howe, Zivich, Gonsalves and Westreich traced the same argument through the literature in 2025.
The size of the distortion is contested too, and it depends on the mechanism rather than on the amount of missing data. Kristman, Manno and Côté simulated 1,000 replications of a cohort of 500 observations and found no important bias at losses to follow-up from 5% to 60% where the loss was missing completely at random or missing at random, and seriously biased odds ratios at low levels of loss where it was missing not at random.
The correction is not free either. Puhani surveyed Monte Carlo studies of Heckman's two-step estimator and concluded that where collinearity is absent "the full-information maximum likelihood estimator is preferable to the limited-information two-step method", and that where collinearity is present, subsample ordinary least squares or the two-part model is the most robust of the estimators that are simple to calculate.
Each of these is named in this entry's own prose as something it gets taken for. Where the two came out differently, that is worth knowing before you quote either.
The first error is a translation error between fields. Hernán records that econometrics and statistics have historically used selection bias for something "essentially equivalent to the epidemiologic term confounding", and Lu, Howe, Zivich, Gonsalves and Westreich write that in clinical trials and economics the term "is commonly used to refer to what epidemiologists define as confounding". Heckman's censored wage sample and Hernán's collider are not the same object, and a reader crossing between the two literatures has to check which one a paper means.
The second error is the one that follows from the first, because the remedies point in opposite directions. A confounder is handled by adjusting for it. A collider is turned into a bias by adjusting for it, so the reflex that fixes one causes the other. Berkson's paradox, entry 58, is the case where that goes wrong inside a hospital register, and it is one species of selection bias rather than a synonym for it.
Survivorship bias, entry 8, is another species, with survival as the filter and the cases that did not last long enough to be counted as the ones missing from the data. The third error is reading a low response rate as a measure of bias. The 1936 Literary Digest poll is the standing example of a poll destroyed by who replied, and its response rate was not the small number the retelling implies.
Lohr and Brick record ten million ballots mailed, almost 2.4 million returned and a response rate of 24 per cent, which is "much higher than the response rate in many of today's polls". The two papers that examined it most closely differ by a point on its headline number: Lohr and Brick report the poll as predicting 54 per cent of the popular vote for Landon, and Squire, printing the magazine's final count of 1,293,669 for Landon, 972,897 for Roosevelt and 83,610 for Lemke, puts Landon's share at 55 per cent.
Roosevelt took more than 60 per cent of the vote and 523 electoral votes. Squire went to a Gallup survey of May 1937 that asked people whether they had received a Literary Digest ballot and whether they had returned it. Among the 780 who said they received one, Roosevelt led 55 to 44; among the 493 who said they returned it, Landon led 51 to 48.
His rough decomposition puts about 11 points of the error on the sample and about 7 on the responses, and he notes that the Gallup poll overstates the winner by 5 points and that too many people claimed to have returned a ballot. Lohr and Brick show that reweighting the magazine's published state-by-state returns by respondents' reported 1932 vote moves ten states, including California and New York, and turns an unweighted prediction of 16 states and 161 of 531 electoral votes for Roosevelt into 26 states and 276, the right winner from the wrong sample.
Their verdict on the magazine is that its real failure was not the response rate it published but that it "failed to assess the uncertainty this implied about their estimates". The fourth error is treating the label as a finding. Naming a selection process establishes that an association might have been manufactured. It does not give the direction or the size, and both still have to be argued for from the data.
Every claim on this page was held against these sources on 2026-08-17. Nothing here is written from memory. Written and checked by Krishna Chagti; how entries are written and corrected. Found a mistake? Report it, with the sentence and a source.