Model training is the process of using data and an optimisation algorithm to adjust the parameters of a machine-learning model so that it learns to perform a specified task or represent useful patterns in the training data. A model begins with parameters whose values may be random or inherited from an existing model.
Training repeatedly presents examples or batches of data, calculates how well the model's predictions correspond to the training objective, computes gradients of a loss function, and updates parameters using an optimisation algorithm. Through many iterations, the model's parameters are adjusted so that the resulting model performs the intended task more effectively according to the chosen objective and data.
The training process normally involves a model, dataset, objective or loss function, optimiser, and computational resources. The loss function quantifies some discrepancy between the model's output and the desired training target. Backpropagation calculates gradients showing how changes in parameters affect the loss, while optimisation algorithms such as stochastic gradient descent and Adam use those gradients to update parameters. Training data are usually divided into batches, and repeated passes through the dataset are called epochs in conventional supervised-learning terminology. Large neural networks can involve billions of parameters, making training computationally intensive and requiring specialised hardware such as GPUs, TPUs, or other accelerators.
Different learning paradigms use different forms of training data and objectives. Supervised learning uses labelled examples, such as inputs paired with correct classifications or outputs. Unsupervised learning seeks patterns or representations without explicit target labels. Self-supervised learning creates training signals from the data itself; language-model pretraining commonly predicts missing, masked, or subsequent tokens according to the architecture and objective. Reinforcement learning uses interactions with an environment and reward signals to optimise behaviour. Modern AI systems can combine several training stages, such as large-scale pretraining followed by supervised fine-tuning and preference or reinforcement-based optimisation.
In large language models, pretraining commonly involves processing enormous collections of text or multimodal data and optimising a predictive objective. A decoder-style language model may be trained to predict the next token given preceding tokens. Repeating this task across very large datasets encourages the model to learn statistical relationships involving syntax, semantics, facts, code patterns, and other regularities present in the training data. The model does not simply store a conventional copy of the dataset; instead, optimisation adjusts its parameters to represent patterns useful for predicting the training objective. The resulting representations can then support downstream tasks that were not explicitly specified as separate supervised objectives during pretraining.
Training must distinguish between fitting the training data and generalising to new data. A model that performs extremely well on examples it has seen but poorly on unseen examples may be overfitting. Validation and test datasets are therefore commonly used to evaluate performance beyond the training set. The validation set can guide hyperparameter and model-selection decisions, while a held-out test set provides a less biased estimate of final performance when used appropriately. Data leakage can undermine these evaluations if information from evaluation sets enters training or influences model development improperly. High training accuracy alone is therefore insufficient evidence that a model will perform reliably in real-world conditions.
Hyperparameters are settings chosen by the practitioner rather than learned directly in the same way as model parameters. Examples include learning rate, batch size, number of training steps or epochs, regularisation strength, architecture dimensions, optimiser configuration, and data-sampling strategy. Training also involves decisions about data cleaning, deduplication, sampling, augmentation, sequence length, precision, checkpointing, and distributed computation. Large-scale training may use data parallelism, tensor parallelism, pipeline parallelism, or combinations of these methods to distribute computation across many accelerators. These engineering choices can have significant effects on training efficiency, stability, cost, and final model quality.
Model training has important practical limitations. Training data can contain errors, bias, copyrighted material, sensitive information, or undesirable patterns, and the resulting model can reflect or amplify aspects of those data. Training can also be expensive in terms of computation, energy, storage, and engineering effort. Reproducibility can be difficult when datasets, software versions, hardware, random seeds, and distributed-training conditions differ. Safety and reliability therefore require more than minimising a loss function. Data governance, evaluation, security, privacy controls, robustness testing, model monitoring, and deployment safeguards may all be required depending on the application and risk level.
Training should be distinguished from inference. Training changes model parameters through optimisation, whereas inference uses fixed learned parameters to produce outputs for new inputs. Fine-tuning is a form of additional training performed on an existing model. Pretraining is an earlier broad training stage intended to establish general capabilities or representations. Transfer learning uses knowledge acquired from one task or dataset to improve performance on another. Training, fine-tuning, prompting, retrieval, and inference therefore represent different mechanisms in an AI system. Understanding these distinctions is essential when assessing what can be changed by modifying data, prompts, external knowledge sources, or model parameters.
