Statistical significance is a concept used in statistical hypothesis testing to describe how incompatible observed data are with a specified null hypothesis under a particular statistical model and set of assumptions. A result is often called statistically significant when its p value is below a preselected significance level, commonly 0.05, although other thresholds can be used and some fields discourage rigid threshold based interpretation.
A p value is the probability, assuming the null hypothesis and model are correct, of obtaining data at least as incompatible with the null hypothesis as the observed data according to the test statistic. It is not the probability that the null hypothesis is true, and it is not the probability that the observed result occurred by chance in the everyday sense. Statistical significance therefore describes a relationship between data, a null model and an inferential procedure rather than establishing scientific importance or causation.
Hypothesis testing begins with a null hypothesis and an alternative hypothesis or alternative set of explanations. The null hypothesis might state that two population means are equal, that a correlation is zero or that a treatment has no specified effect. Researchers calculate a test statistic from the observed data and determine how unusual that statistic would be under the null model. The p value quantifies this extremeness according to the chosen test.
If the p value falls below the prespecified significance level, researchers may reject the null hypothesis under the rules of that test. If it does not, the appropriate conclusion is generally that the evidence is insufficient to reject the null rather than that the null has been proved true. This distinction is important because failure to detect an effect can result from low statistical power, noisy measurements, small sample size or an effect that is genuinely absent.
Statistical significance depends strongly on sample size. With a sufficiently large sample, a very small effect can produce a small p value because the estimate becomes precise. Conversely, a substantial effect can fail to reach a conventional significance threshold when the sample is small or variability is high. This is why statistical significance should be distinguished from effect size.
Effect size describes the magnitude of an observed or estimated difference or association, while statistical significance concerns evidence relative to a null model and sampling variability. Confidence intervals provide additional information by showing a range of parameter values compatible with the data and assumptions at a specified confidence level. Reporting effect sizes and uncertainty alongside p values generally provides a more informative description than reporting statistical significance alone.
Statistical significance does not measure practical or scientific importance. An intervention can have a statistically significant effect that is too small to matter in practice, while an important effect can remain statistically uncertain because available data are limited. Practical significance depends on the context, costs, risks, benefits and magnitude of the effect relative to meaningful thresholds. Clinical research, for example, often distinguishes statistical evidence of a difference from whether the difference is large enough to be clinically meaningful.
Engineering may compare an estimated performance improvement with operational tolerances, while environmental science may assess whether a measured change is ecologically consequential. The interpretation of statistical significance therefore requires subject matter knowledge and should not be reduced to whether a p value crosses one conventional boundary.
Multiple testing creates another important issue. If researchers test many hypotheses independently using a conventional threshold, some tests may produce statistically significant results purely because of the number of opportunities for a low p value to occur. For example, testing one hundred independent null hypotheses at a 0.05 threshold can produce some apparently significant results even when all null hypotheses are true.
Methods such as Bonferroni correction, false discovery rate procedures and prespecified analysis plans can address aspects of this problem. Selective reporting, optional stopping, trying many analytical specifications and reporting only favourable results can also increase the frequency of apparently significant findings. These practices are among the reasons preregistration, transparent reporting and replication can improve the reliability of statistical inference.
Statistical significance is also affected by the assumptions of the statistical model. Tests can rely on assumptions concerning independence, distributional form, variance, measurement scale or sampling design. Violations of these assumptions can alter p values and confidence intervals. Some methods are robust to particular departures, while others require alternative models or transformations. The sampling process matters because classical inferential statistics generally assumes a relationship between the observed sample and a defined population or data generating process. A statistically significant result from a severely biased sample may not support the intended population claim. Statistical inference therefore cannot be separated entirely from research design, measurement quality and the process by which observations were obtained.
Modern statistical practice increasingly emphasises estimation, uncertainty and the complete analytical context rather than treating a significance threshold as a binary decision rule. Confidence intervals, likelihood based inference, Bayesian methods, model comparison and prediction intervals provide alternative or complementary approaches. Bayesian inference, for example, combines prior information with observed data to produce a posterior distribution and does not use statistical significance in the classical p value sense.
This does not make Bayesian analysis automatically superior or inferior; it answers inferential questions using a different framework. Frequentist hypothesis testing itself contains many legitimate approaches beyond a simple p value threshold. The central issue is matching the statistical method to the scientific question and being explicit about assumptions and uncertainty.
Statistical significance is therefore best understood as one component of evidence rather than a universal measure of truth. A statistically significant result indicates that the observed data would be relatively unusual under the specified null hypothesis and model, according to the chosen test. It does not establish that the effect is large, important, causal, reproducible or free from bias.
A complete interpretation should consider effect size, confidence intervals or other uncertainty measures, sample size, study design, measurement quality, multiple testing, model assumptions, prior evidence and the scientific context. Statistical significance can be useful when applied appropriately, but it becomes misleading when treated as a simple label separating true findings from false findings.
