Polling bias is a systematic tendency for a survey's results to differ from the underlying value in the target population because of features of sampling, recruitment, measurement, weighting, analysis or other parts of the research process. Unlike random sampling error, which can cause an estimate to fluctuate above or below the population value from one sample to another, bias refers to a directional or systematic distortion.
In political polling, bias can arise if certain groups are consistently less likely to be included, if respondents who participate differ from those who do not, if a sampling frame excludes relevant people, if question wording encourages particular interpretations, or if an analytical model makes inappropriate assumptions. A poll can therefore have a small reported margin of error while still suffering from meaningful bias. AAPOR distinguishes sampling error from other errors and stresses that nonresponse and methodological choices can affect results.
Sampling bias occurs when the mechanism used to select respondents systematically favours some members of the target population over others in a way that is related to the subject being measured. Coverage bias is closely related and occurs when parts of the target population are inadequately covered by the sampling frame. Self selection can create another form of bias when people who voluntarily participate differ from people who do not.
A call in political poll, for example, can attract people who are unusually motivated to express an opinion, while an opt in online survey can attract respondents with characteristics associated with willingness to join research panels. AAPOR warns that self selected samples can differ from the wider population and that such differences can affect political estimates.
Nonresponse bias occurs when selected people fail to participate and the nonrespondents differ systematically from respondents in ways relevant to the survey. A low response rate does not automatically prove that a poll is biased, because the relationship between response rate and bias is not mechanically one to one. What matters is whether the people who respond differ from those who do not in characteristics related to the measured outcomes.
Conversely, even a survey with a substantial response rate can be biased if its recruitment process systematically excludes particular groups. Pollsters may use weighting or other adjustments to compensate for observed differences, but these procedures depend on available information and assumptions. AAPOR specifically identifies nonresponse as a major factor affecting polling accuracy and explains that even probability samples can be affected when respondents and nonrespondents differ systematically.
Measurement bias arises when the survey instrument systematically influences the answers being measured. Question wording is one of the principal mechanisms. A politically neutral question and a question containing emotionally charged or leading language may produce different distributions of responses even when asked to the same population. Question order can also influence results because earlier questions can establish a context that affects how later questions are interpreted.
Response categories, definitions, question length and assumptions embedded in the wording can similarly affect measurement. AAPOR states that wording and sequence can influence poll results and recommends transparency about questionnaire construction. Measurement bias is therefore distinct from sampling bias because it can occur even when the sample itself is well selected.
Weighting can reduce some forms of observed sample imbalance but can also interact with polling bias in complex ways. If a sample contains too many respondents from one demographic group and too few from another, weights can alter the contribution of respondents so that the final estimates better match known population benchmarks. However, weighting only adjusts the dimensions included in the model and cannot automatically correct unknown differences. Large weights can also increase variance and reduce the effective sample size. AAPOR recommends disclosure of the variables and procedures used in weighting, including adjustments for selection probabilities, eligibility, nonresponse, post stratification, raking and calibration. This illustrates why a poll's final weighted percentages cannot be understood properly without knowing how the weights were constructed.
Political polling bias can also emerge from assumptions about turnout. A pre election poll may begin with a sample of adults or registered voters and then attempt to estimate the subset most likely to vote. If the turnout model systematically overestimates participation by one group or underestimates participation by another, the resulting candidate or party estimates can be biased even if the underlying survey respondents were accurately sampled. This is particularly important because actual turnout is not known before the election. AAPOR identifies likely voter modelling as one of the major challenges of election polling and notes that underestimating or overestimating turnout among particular groups can alter poll estimates.
Polling bias should not be confused with deliberate manipulation. A biased result can arise from ordinary methodological problems, imperfect information, changing communication patterns or difficult statistical decisions without any intention to produce a particular political outcome. Pollsters must make choices about sampling frames, recruitment, weighting, likely voters, question order and data collection modes, and those choices can produce different estimates. AAPOR notes that legitimate polling organisations can make methodological choices that shift results without those choices constituting fabrication. The correct analytical approach is therefore to investigate the design and evidence rather than infer motive from a difference between a poll and an eventual result. Bias is a methodological property of an estimate, not automatically evidence of intentional misconduct.
Polling bias is best understood as one component of total survey error and should be evaluated alongside sampling variation, measurement error, nonresponse, coverage, weighting and timing. A single poll can be biased in one direction while another poll has different methodological weaknesses, meaning that disagreement between polls does not automatically reveal which survey is correct. Comparing methods, fieldwork dates, sample sources and weighting procedures can help identify plausible explanations for differences. Polling averages may reduce some random noise across surveys, but they can still inherit common systematic biases shared by the underlying polls. AAPOR therefore recommends methodological transparency and cautions that polls are approximate measurements rather than infallible representations of political reality.
