Transfer Learning is a machine-learning approach in which knowledge, representations, parameters or learned structures obtained from one task, dataset or domain are reused to improve learning or performance on another related task or domain. The source and target problems need not be identical, but useful common structure must exist for transfer to provide an advantage.
A common modern example involves pretraining a neural network on a large dataset and then adapting it to a downstream task using a smaller amount of task-specific data. Transfer can involve reusing a complete pretrained model, initialising a new model with learned parameters, extracting fixed representations, fine-tuning selected layers or adapting a model using parameter-efficient techniques. The fundamental motivation is to avoid learning everything from scratch when prior training has already captured information relevant to the target problem. Transfer learning therefore concerns the movement of learned information between learning problems rather than one particular algorithm or neural-network architecture.
The source and target domains can differ in their input distributions, tasks or both. Domain adaptation is a related area concerned particularly with differences between source and target domains when the task may remain similar. If the source and target tasks differ substantially, more general transfer-learning techniques may be required. Feature-based transfer uses representations learned from the source problem as inputs to a target model. Parameter transfer uses source-trained parameters as an initialisation for target training.
Fine-tuning continues optimisation on target data, potentially updating the entire model or selected components. In large neural networks, transfer can occur through pretrained embeddings, shared encoders, adapters, low-rank parameter updates or other mechanisms. The usefulness of transfer depends on the similarity and compatibility of the learned information with the target problem, and indiscriminately transferring a model does not guarantee improvement.
A common transfer-learning workflow begins with pretraining on a source dataset or broad corpus, followed by adaptation to the target task. Early neural-network layers in some architectures may learn relatively general representations, while later layers can become more task-specific, although this is not a universal rule. Practitioners may freeze selected layers so that only a small portion of the model is updated, reducing computational requirements and the amount of target data needed to modify parameters.
Full fine-tuning can provide greater adaptation when sufficient target data and computation are available. Parameter-efficient fine-tuning methods instead introduce or update a comparatively small set of parameters while keeping most pretrained weights fixed. These approaches are particularly useful when one base model must support many downstream tasks or users without storing a complete independently fine-tuned model for every application.
Transfer learning is central to contemporary natural language processing, computer vision and speech processing. A language model can be pretrained on broad text and adapted to document classification, information extraction or other specialised tasks. An image model trained on a large visual corpus can provide representations that support a smaller target dataset. Speech representations learned from extensive audio can be adapted to particular languages or specialised acoustic conditions.
Scientific and medical applications can similarly use pretrained models when target-domain datasets are limited, although domain differences can be substantial. The principal benefit is that the target model begins with information learned from a much larger or otherwise useful source dataset. This can improve sample efficiency, reduce training time and sometimes increase performance compared with random initialisation, although the magnitude and even direction of the benefit depend on the source-target relationship.
Negative transfer occurs when knowledge acquired from the source problem harms performance on the target problem. This can happen when source and target distributions differ substantially, when the learned features encode misleading assumptions, or when optimisation becomes trapped around an unsuitable initialisation. A source model can also contain biases or undesirable correlations that transfer into the target system. Catastrophic forgetting is another concern during adaptation: extensive fine-tuning on a new task can degrade capabilities acquired during pretraining. The degree of parameter updating, learning rate, target dataset size and adaptation method all influence this trade-off. Evaluation should therefore compare transfer-based training against appropriate baselines, including training from scratch when feasible, rather than assuming that pretraining necessarily provides an advantage.
Transfer learning differs from self-supervised learning because the two describe different aspects of the training process. Self-supervised learning specifies how a learning signal can be generated from unlabelled data, whereas transfer learning specifies that learned information from one problem is reused for another. A model can therefore undergo self-supervised pretraining and subsequently be transferred to a downstream task.
Transfer learning also differs from federated learning: transfer concerns reuse across tasks or domains, while federated learning concerns distributed training across data holders or devices. Reinforcement learning can use transfer when policies, representations or value functions are reused across environments. Semi-supervised learning can also benefit from transferred representations. These relationships show that transfer learning is compatible with several training paradigms and should not be treated as a competing alternative to them.
The advantages of transfer learning include reduced target-data requirements, shorter training times, reuse of expensive pretraining and the possibility of achieving strong performance on specialised tasks without constructing a model entirely from scratch. Its limitations include negative transfer, source-data bias, domain mismatch, computational costs associated with large pretrained models and potential incompatibility between source and target objectives. The licensing, privacy and provenance of source training data can also matter when pretrained models are reused commercially or across organisations.
Model versioning is important because changes to the source model can alter downstream performance. Security considerations also arise if pretrained models contain vulnerabilities or malicious behaviours introduced through compromised training or distribution pipelines. Transfer learning therefore provides an efficiency mechanism, but its effectiveness depends on careful selection, validation and adaptation of the transferred knowledge.
Transfer Learning is best understood as a mechanism for leveraging accumulated learning. Its significance has increased as modern models have become large enough to encode broad reusable representations and as self-supervised pretraining has enabled learning from enormous unlabelled datasets. The approach can range from simple reuse of fixed features to extensive adaptation of a pretrained foundation model.
The term does not imply that the source and target tasks must be identical, nor does it imply that all learned information is beneficial. The central question is whether information learned in the source setting contains structure that can reduce the difficulty of the target problem. Successful transfer can improve data efficiency and training efficiency, but sound evaluation must establish that the transferred knowledge actually benefits the target task under realistic deployment conditions.
