Fine-tuning is the process of taking an already trained machine-learning model and continuing its training on a narrower or task-specific dataset so that its parameters adapt to the desired behaviour or domain. Instead of training the model entirely from randomly initialised parameters, fine-tuning starts from an existing pretrained model. The additional training can teach specialised patterns, output formats, terminology, instruction-following behaviour, classification boundaries, or other task-specific capabilities. In language-model development, fine-tuning is commonly performed after broad pretraining and may be followed by additional alignment or preference-optimisation stages.
Fine-tuning works by presenting training examples to the existing model and using an optimisation procedure to adjust some or all of its parameters. The learning rate is typically smaller than in initial pretraining because the objective is to adapt rather than completely relearn the model. Training data quality is especially important because fine-tuning can reinforce undesirable patterns, biases, incorrect information, or unwanted formatting if the examples contain them. The model's previous capabilities can also be affected if the new training is too narrow or aggressive. The process therefore requires careful dataset construction, hyperparameter selection, evaluation, and monitoring for unintended changes.
There are several forms of fine-tuning. Full fine-tuning updates essentially all trainable model parameters, which can require substantial computational resources for large models. Parameter-efficient fine-tuning methods update only a small number of additional or selected parameters. Low-Rank Adaptation, commonly known as LoRA, introduces trainable low-rank matrices while leaving most original weights frozen. Other techniques include adapters, prefix tuning, prompt tuning, and related methods. Quantisation-aware or quantised parameter-efficient approaches can reduce memory requirements further. These techniques make specialisation more practical for large models without requiring a complete copy of all model parameters for every task.
Fine-tuning is different from prompting. Prompting changes the input supplied to a model without modifying its learned parameters. Fine-tuning changes model parameters through additional training. Prompting can therefore be quick and inexpensive to iterate, while fine-tuning can produce more persistent behavioural changes. Fine-tuning is also different from RAG. RAG supplies external information at inference time; fine-tuning modifies the model itself. If the objective is to provide constantly changing factual information, RAG may be more appropriate than repeatedly fine-tuning a model. If the objective is to make a model consistently produce a particular format or follow a specialised style or task pattern, fine-tuning may be useful.
Fine-tuning does not necessarily function as a reliable database-loading mechanism. Training a model on a collection of factual documents can alter its statistical behaviour, but it does not guarantee that every fact will be memorised accurately, remain accessible, or be updated predictably. Fine-tuned models can still hallucinate, and updating a frequently changing knowledge base through repeated fine-tuning can be operationally cumbersome. For knowledge retrieval, external databases or RAG systems often provide more direct update and provenance mechanisms. Fine-tuning can instead be valuable for teaching how to perform a task, how to structure outputs, how to classify inputs, or how to behave according to a stable specialised pattern.
The quality and composition of the fine-tuning dataset strongly affect results. Examples should represent the desired inputs, outputs, edge cases, terminology, and constraints. If the dataset is too small, noisy, repetitive, or narrowly distributed, the model may overfit or fail to generalise. If examples conflict with one another, the resulting behaviour may become inconsistent. Evaluation should use held-out examples and, where possible, real-world representative cases that were not included in training. Tests should measure both the target behaviour and important capabilities that must not regress, because improving one specialised metric can sometimes reduce performance elsewhere.
Fine-tuning can be used for many tasks, including text classification, structured extraction, instruction following, domain-specific assistance, code generation, summarisation, style control, and specialised conversational behaviour. In classification, examples can teach a model to map inputs to predefined labels. In structured generation, examples can demonstrate required schemas and formatting. In domain adaptation, fine-tuning can expose the model to specialised terminology and recurring patterns. The exact usefulness depends on the base model, task, dataset size, quality, and evaluation methodology. Fine-tuning is not inherently superior to prompting or retrieval; it is one tool among several adaptation techniques.
Fine-tuning has costs and risks. Training consumes compute and requires dataset preparation, experiment tracking, evaluation, model versioning, and deployment infrastructure. Poorly selected data can encode undesirable behaviour, and models can overfit. There can also be catastrophic forgetting or loss of general capabilities when adaptation is poorly configured, although the severity depends on the method and training regime. Version management is important because a fine-tuned model becomes a distinct model artefact that must be evaluated when its base model, training data, or training configuration changes. Effective fine-tuning therefore involves controlled experimentation and rigorous comparison against an appropriate baseline rather than assuming that additional training automatically improves a model.
