Deep Learning is a branch of machine learning based primarily on neural networks with multiple layers of learned transformations. The term refers to the depth and hierarchical composition of representations rather than to one specific neural-network architecture. A deep model can transform an input through successive computational layers, with intermediate representations becoming increasingly adapted to the task.
In image recognition, early layers may learn local visual patterns while later layers represent more complex structures; in language processing, layers can learn increasingly contextual representations of tokens and sequences. Deep learning became especially influential because large neural networks can learn useful representations from relatively raw inputs, reducing the need for manually engineered features in many applications. Its success has been driven by the interaction of large datasets, improved optimisation techniques, specialised hardware, architectural innovations and increased computational scale.
The mathematical basis of a neural network is a composition of parameterised functions. A basic neuron computes a weighted combination of inputs, adds a bias and applies an activation function. Layers of such units can be composed to form nonlinear functions with substantial representational capacity. During training, an objective function measures discrepancies between model outputs and desired or self-derived targets, and backpropagation computes gradients that indicate how parameter changes would affect the objective.
An optimisation algorithm such as stochastic gradient descent or a related method then updates the parameters. Deep networks may contain millions, billions or more trainable parameters, although parameter count alone does not determine capability. Activation functions, normalisation, residual connections, attention mechanisms, regularisation and architectural structure all affect optimisation and representation. The chain-rule-based computation of gradients through multiple layers is fundamental to the practical training of modern neural networks.
Several architectures have played important roles in deep learning. Convolutional neural networks were particularly influential in computer vision because convolutional operations exploit local spatial structure and parameter sharing. Recurrent neural networks and related gated architectures were widely used for sequential data because they process information across time steps, although they have increasingly been supplemented or replaced in many applications by attention-based approaches.
Transformers use attention mechanisms to model relationships among elements of a sequence or other structured input and have become central to modern language and multimodal systems. Autoencoders learn representations through reconstruction objectives, while generative adversarial networks train competing generator and discriminator networks. Diffusion models learn processes that can generate data through iterative denoising and have become important in image, audio and other generative applications. These architectures are not interchangeable: their computational structures impose different inductive biases, resource requirements and application characteristics.
Deep learning is used in image classification, object detection, semantic segmentation, speech recognition, machine translation, text generation, information extraction, recommendation, scientific modelling and many other tasks. A computer-vision model may process pixels directly to identify objects or medical image features. A speech model can transform an acoustic signal into text or other representations. A language model can learn statistical relationships among tokens and generate or classify text.
Multimodal models can process combinations of text, images, audio or other modalities, depending on their architecture and training data. Deep reinforcement learning combines deep networks with reinforcement-learning objectives to approximate policies, value functions or other components required for decision-making. The broad applicability of deep learning comes partly from its ability to learn high-dimensional nonlinear representations, but practical success still depends on suitable data, objectives, architecture, optimisation and evaluation.
The modern deep-learning expansion is closely associated with the availability of graphics processing units and other accelerators capable of performing large numbers of parallel numerical operations. Large-scale training also depends on distributed computing, efficient data pipelines, high-capacity storage and software frameworks for automatic differentiation and hardware acceleration. Model scaling can increase computational requirements substantially, and training large models may involve parallelism across multiple devices or machines.
Inference can also be expensive when models are large or when applications require low latency. Techniques such as quantisation, pruning, knowledge distillation, sparsity and efficient architectures can reduce deployment costs in suitable circumstances. Memory bandwidth, communication overhead and accelerator availability can become bottlenecks independently of raw arithmetic throughput. Deep learning is therefore both an algorithmic discipline and an engineering problem involving hardware, software, data and systems design.
Deep models can exhibit powerful generalisation but are also capable of overfitting, shortcut learning and sensitivity to changes in input distributions. Shortcut learning occurs when a model exploits a predictive feature that performs well in training conditions but does not represent the intended underlying concept. Adversarial examples demonstrate that small, carefully constructed input changes can sometimes cause incorrect predictions without being obvious to humans.
Neural networks can also encode biases present in training data, reproduce unwanted correlations and produce outputs with poorly calibrated confidence. Interpretability is an active research area because the relationship between individual parameters, internal representations and final behaviour can be difficult to characterise. Techniques such as saliency analysis, feature attribution, probing, activation analysis and mechanistic interpretability provide different forms of evidence, but none should automatically be treated as a complete explanation of a model's behaviour.
Deep learning offers major advantages when relationships are high-dimensional, nonlinear or difficult to capture through manually engineered features. End-to-end training can allow a system to optimise representations jointly with its final task objective, and pretraining can produce reusable representations that support many downstream applications. However, deep models often require substantial data and computation, can be difficult to debug, and may exhibit unpredictable behaviour outside their training distribution.
Large models can also contain redundant parameters and may incur significant energy, hardware and financial costs. Their outputs must be evaluated according to the intended application, because impressive benchmark performance does not establish universal reliability. Model compression, transfer learning and parameter-efficient adaptation can reduce some resource requirements, but trade-offs between accuracy, efficiency, adaptability and robustness remain important engineering considerations.
Deep Learning is a specific methodological family within Machine Learning rather than an alternative to it. Its relationship with the other learning paradigms depends primarily on how the neural network is trained. Deep networks can learn from labelled examples through supervised learning, construct their own prediction targets through self-supervised learning, use mixtures of labelled and unlabelled data through semi-supervised learning, adapt pretrained representations through transfer learning, or learn action policies through reinforcement learning.
Federated learning can also be applied to deep neural networks when training data remain distributed across participating devices or organisations. This means that the terms describe different dimensions of a system: deep learning primarily concerns the model architecture and representation-learning approach, while many of the other terms describe the source or organisation of the learning signal and the circumstances under which training occurs.
