What is a Representative Sample?

Politics — James William Steven Parker

A representative sample is a sample designed and selected so that the information obtained from the sampled respondents can provide a credible basis for estimating characteristics of a defined target population. In political polling, representativeness means that the people included in the survey, after taking account of the sampling design and any appropriate weighting, correspond sufficiently to the population on characteristics relevant to the question being studied.

It does not mean that every individual in the population must be present, nor does it mean that the sample must be an exact miniature copy of the population in every respect. A representative sample is fundamentally about the validity of inference from the observed respondents to the specified population. The meaning of representative therefore depends on what population is being studied and what characteristics matter for the particular research question.

Representativeness can be approached through probability sampling, careful recruitment, demographic controls and statistical weighting. Probability sampling gives eligible members of the target population known chances of selection and provides a formal basis for statistical inference. However, even a probability sample can become less representative in practice if coverage is incomplete or selected people do not respond. Pollsters therefore often compare the composition of respondents with known population benchmarks and apply weights to adjust differences.

Weighting can make respondents from underrepresented groups contribute more heavily and respondents from overrepresented groups contribute less heavily. AAPOR describes weighting as a statistical technique used to make the relative contribution of respondents correspond more closely to population characteristics such as age, sex, race or ethnicity, location, educational attainment and political identification.

A representative sample should always be understood relative to a target population. A sample can be representative of adults but not of registered voters, representative of registered voters but not of likely voters, or representative of a national population but not of a particular constituency. The target population must therefore be specified before representativeness can be assessed. This is especially important in election polling because turnout is selective.

People who are eligible to vote are not necessarily the same people who actually vote, and people who say they intend to vote do not all necessarily participate. Pollsters may therefore construct a likely voter model or another turnout framework to estimate the electorate. AAPOR identifies turnout modelling as a major challenge because the actual voting population is not fully known until the election takes place.

Representativeness is not synonymous with matching demographic percentages alone. A sample may closely match the population in age, gender and geography while differing in political engagement, media use, turnout probability, party identification or other characteristics relevant to the subject being measured. Weighting can only adjust characteristics that are measured and for which suitable population benchmarks or modelling assumptions are available.

If an unmeasured characteristic is related to both participation in the survey and political opinion, residual bias can remain after weighting. This is one reason why survey methodology cannot be reduced to a simple demographic checklist. AAPOR emphasises that nonresponse and self selection can affect results because participants may differ systematically from people who do not participate, even when the initial sampling process is sound.

The concept of a representative sample is closely related to probability sampling but the terms are not identical. Probability sampling is a method of selecting respondents in which selection probabilities are known, whereas representativeness concerns whether the resulting evidence provides a credible basis for inference about the target population. A well designed probability sample can still suffer from nonresponse or coverage problems, while a non probability sample can sometimes be adjusted using statistical models to resemble known population characteristics.

However, model based adjustment does not create the same formal statistical foundation as probability sampling. AAPOR therefore cautions against automatically applying a conventional margin of sampling error to non probability samples, because the classical calculation describes uncertainty arising from probability based sample selection and does not capture all uncertainties created by non probability recruitment.

A representative sample also depends on the quality of the sampling frame. A sampling frame is the source from which potential respondents are identified. If the frame excludes a substantial segment of the target population, the resulting sample may have a coverage problem before recruitment even begins. Modern polling can use several types of frames, including telephone based frames, address based samples, voter files, membership lists and online panels, depending on the population and research objective.

Different frames have different strengths and limitations. AAPOR's disclosure standards call for pollsters to identify whether a sample is probability based or non probability based and to describe the frame, its coverage of the target population and any segments that are not covered by the design.

Representativeness is also affected by how questions are asked and how respondents behave. A sample can accurately reflect the population of people who answered a question while the question itself measures the intended political concept poorly. Respondents can misunderstand wording, use response options differently, decline particular questions or provide answers influenced by social pressures. Consequently, representativeness concerns the relationship between the sample and population, whereas measurement validity concerns whether the survey instrument accurately measures the intended concept.

A high quality political poll needs both. A perfectly balanced sample cannot rescue a badly constructed questionnaire, and a well written questionnaire cannot compensate for a systematically unrepresentative sample. AAPOR identifies both sampling procedures and question wording as important factors affecting the validity of survey findings.

The practical significance of a representative sample is that it permits researchers to make population level estimates without questioning every member of the population. The goal is not perfect reproduction of reality but defensible statistical inference under stated assumptions. Readers should therefore avoid treating the word representative as a guarantee of accuracy. A representative design can still contain sampling uncertainty, nonresponse effects, measurement error and other sources of error.

AAPOR explicitly notes that polls can differ from eventual election results because of sampling error, changes in preferences, difficulties recruiting certain groups and uncertainty about turnout. Representativeness is consequently an important property of a poll sample, but it is one component of overall survey quality rather than a guarantee that every published estimate will match the eventual population value.