Statistics is the discipline concerned with collecting, organising, analysing, interpreting and communicating data in order to describe observed phenomena and make inferences about processes or populations that are not completely observed. Its mathematical foundations include probability theory, mathematical optimisation, linear algebra and calculus, while its practical methods range from simple counts, percentages and averages to regression, hypothesis testing, experimental design, Bayesian inference and multivariate analysis.
Descriptive statistics summarise the information contained in an observed dataset, whereas inferential statistics use samples to make statements about a broader population while explicitly accounting for sampling variability and uncertainty. Statistics is used across science, medicine, engineering, economics, government, business, social research and many other fields because empirical observations generally contain variation rather than producing identical outcomes.
A central distinction in Statistics is between a population, which represents the complete set of units or measurements relevant to a question, and a sample, which is a subset observed or measured from that population. Variables may be quantitative, such as age, temperature or income, or categorical, such as species, treatment group or geographical region. Quantitative variables can be discrete or continuous, and categorical variables can be nominal or ordinal.
Statistical summaries include the mean, median, mode, variance, standard deviation, range, quantiles and proportions. The mean is sensitive to extreme observations, while the median is generally more resistant to skewed distributions and outliers. Measures of spread complement measures of central tendency because two datasets can have the same mean but substantially different variability.
Statistical inference addresses uncertainty rather than eliminating it. Sampling distributions describe how an estimator would vary across repeated samples, while standard errors quantify the estimated variability of many estimators. Confidence intervals provide ranges constructed according to a specified statistical procedure, and hypothesis tests evaluate data relative to a null hypothesis under stated assumptions.
A small p-value does not by itself establish that a hypothesis is false or that an effect is practically important, and statistical significance does not imply causal significance. Effect sizes, uncertainty intervals, study design, measurement quality and substantive context are therefore important when interpreting results. Multiple testing, optional stopping, selective reporting and inappropriate model selection can substantially distort inferential conclusions.
Probability provides the conceptual framework underlying much of Statistics. Probability distributions describe possible values and their associated probabilities or probability densities, with common distributions including the Bernoulli, binomial, Poisson, normal, exponential, gamma and Student's t distributions. Expected value describes a probability-weighted average, while variance measures dispersion around the expectation. Conditional probability and independence are particularly important because statistical relationships often depend on which information has already been observed. Bayes' theorem provides a formal method for updating probabilities when new evidence is obtained. Frequentist and Bayesian approaches differ principally in how probability and unknown parameters are interpreted and how inference is constructed, although both can use closely related mathematical models and computational methods.
Regression and related modelling techniques allow Statistics to represent relationships between variables while accounting for uncertainty. Linear regression models a response using linear combinations of predictors and can be extended through interactions, transformations, polynomial terms and regularisation. Logistic regression is commonly used for binary outcomes, while Poisson and negative binomial models can be used for count data under appropriate assumptions.
Multivariate methods analyse several variables jointly, with techniques such as principal component analysis, cluster analysis and discriminant analysis addressing different objectives. Correlation measures association but does not by itself establish causation. Confounding, selection bias, measurement error, reverse causality and inappropriate adjustment can produce misleading statistical relationships even when calculations are performed correctly.
Statistical methodology depends heavily on study design and data-generating mechanisms. Randomised experiments can provide strong evidence about causal effects when properly designed and implemented because random allocation helps balance potential confounders. Observational studies lack that same controlled allocation and therefore often require additional assumptions and methods for causal interpretation. Random sampling concerns how observations are selected from a population, whereas random assignment concerns how experimental treatments are allocated to participants or experimental units; these are distinct concepts. Blocking, stratification, matching, replication and controlled randomisation can improve experimental efficiency or precision. Missing data also require careful treatment because the mechanism producing missingness can affect estimates, particularly when missingness depends on unobserved values or variables related to the outcome.
Statistics has important practical limitations. Statistical conclusions are conditional on the quality and representativeness of the data, the validity of measurement procedures, the appropriateness of the model and the assumptions supporting the inference. A large dataset does not automatically eliminate bias, because systematic selection or measurement errors can persist regardless of sample size. Conversely, small datasets can sometimes provide informative evidence when collected through appropriate designs and analysed with methods suited to their structure. Data leakage, dependence between observations, pseudoreplication, overfitting and unrecognised batch effects are common analytical problems. Reproducibility also depends on preserving data provenance, analytical code, preprocessing decisions, model specifications and sufficient documentation so that results can be independently checked.
Modern Statistics increasingly intersects with computing, machine learning, data science and scientific research. Statistical software allows analysts to perform numerical estimation, resampling, simulation, optimisation and high-dimensional modelling that would be impractical manually. Bootstrap methods estimate sampling uncertainty through repeated resampling, while Monte Carlo methods use random sampling to approximate quantities that are difficult to calculate analytically.
Machine learning often emphasises predictive performance, whereas traditional statistical analysis frequently places greater emphasis on interpretable parameters, uncertainty and data-generating assumptions, although the boundary is not absolute. Effective statistical communication requires presenting results in ways that preserve uncertainty and avoid misleading interpretations. The discipline therefore combines mathematical theory, computational methods, empirical reasoning and careful interpretation rather than being merely the calculation of averages or percentages.
