Artificial Neural Networks are computational models composed of interconnected mathematical units whose parameters are adjusted during training to approximate relationships between inputs and outputs or to learn useful internal representations. The terminology is inspired by biological nervous systems, but Artificial Neural Networks are mathematical and engineering constructs rather than literal simulations of biological neurons. A network typically contains layers of computational units, with each unit applying a weighted transformation and a nonlinear activation function or another specified operation.
Parameters such as weights and biases are learned from data using an optimisation procedure. Networks can perform classification, regression, prediction, generation, feature extraction, control and other computational tasks. Their defining characteristic is not a particular application but the use of interconnected parameterised transformations whose behaviour is learned or adjusted through training.
A basic feed-forward Artificial Neural Network receives an input vector, transforms it through one or more hidden layers and produces an output. Each computational unit calculates a weighted combination of inputs, adds a bias and applies an activation function. Linear transformations alone can represent only limited classes of functions, while nonlinear activation functions enable networks to approximate much more complex relationships.
Common activations have included sigmoid and hyperbolic tangent functions, while rectified linear and related functions became especially important in deep networks because they provide useful optimisation characteristics. The architecture determines how information flows and constrains the types of relationships the model can represent. Output layers are designed according to the task, such as producing class probabilities, continuous values or parameters of a probability distribution.
Training an Artificial Neural Network generally involves defining a loss function that measures the discrepancy between predictions and desired outcomes, calculating gradients of that loss with respect to model parameters and updating the parameters using an optimisation algorithm. Backpropagation efficiently applies the chain rule of calculus to calculate gradients through multiple layers. Optimisers then use those gradients to modify weights and biases.
Training can involve minibatches rather than the entire dataset at every update, allowing large datasets to be processed efficiently. Learning rate, batch size, regularisation, initialisation and optimisation strategy influence convergence and generalisation. Training can be supervised, where target outputs are provided, or use unsupervised, self-supervised, reinforcement or other learning arrangements depending on how the objective is defined.
Deep Artificial Neural Networks contain many layers or large numbers of computational transformations and can learn hierarchical representations from raw or relatively minimally processed data. Earlier neural systems were often constrained by limited computing power, insufficient datasets and optimisation difficulties. Improvements in hardware, algorithms, activation functions, regularisation, data availability and training techniques enabled much larger networks.
Deep learning is therefore strongly associated with modern Artificial Neural Networks, although the terms are not strictly synonymous: deep learning refers to learning with sufficiently deep hierarchical models, whereas Artificial Neural Networks is the broader family. Large networks can learn increasingly abstract representations across layers, although the exact nature of these representations varies by architecture and task.
Artificial Neural Networks encompass numerous architectural forms because different connectivity patterns impose different inductive biases. Fully connected networks connect units across adjacent layers and are useful for many tabular or general-purpose transformations. Convolutional Neural Networks incorporate local connectivity and shared filters, making them particularly effective for spatially structured data. Recurrent Neural Networks introduce recurrent connections for sequential processing. Transformer Models use attention-based interactions to model relationships among elements of a sequence or other structured input.
Autoencoding architectures learn transformations between inputs and latent or reconstructed representations. Generative architectures can learn distributions from which new samples are produced. These architectures should not be treated as interchangeable implementations because their computational structures affect efficiency, representation learning, context handling and suitability for particular data.
Artificial Neural Networks are powerful because sufficiently expressive networks can approximate broad classes of functions, while representation learning allows useful features to be discovered automatically from data. Their advantages include flexibility, strong performance on many complex prediction tasks and the ability to learn from high-dimensional inputs such as images, audio and text. However, expressive capacity creates challenges. A network can overfit its training data, learn spurious correlations or perform poorly when deployment data differ from training data. Large networks may require substantial computational resources and extensive datasets. Their internal representations can also be difficult to interpret directly. Performance metrics can conceal systematic errors, and high benchmark accuracy does not necessarily imply robustness to distribution shifts, adversarial inputs or unusual cases.
Practical deployment requires considerations beyond network architecture. Data preprocessing, training-validation-test separation, hyperparameter selection, regularisation, model calibration, monitoring and hardware requirements all affect system performance. Batch normalisation, dropout, weight decay, data augmentation and early stopping are examples of techniques that can influence training or generalisation, depending on the architecture and task. Inference requirements can differ considerably from training requirements, and techniques such as quantisation, pruning or distillation can reduce deployment cost in appropriate circumstances. Neural networks can also be integrated with symbolic systems, databases, search mechanisms and external tools. Consequently, the behaviour of an Artificial Neural Network in a production system depends on the complete pipeline rather than on the network weights alone.
The historical significance of Artificial Neural Networks lies in the gradual development of computational learning systems capable of representing increasingly complex functions. Early models demonstrated that networks of simple units could perform useful computations, while later multilayer systems established practical gradient-based learning. The resurgence of neural approaches during the deep-learning era resulted from the interaction of large datasets, powerful parallel processors, improved optimisation methods and architectural advances.
Modern Artificial Neural Networks underpin major areas of machine learning, including computer vision, speech processing, language modelling, recommendation, scientific modelling and generative systems. Their continued development has involved increasing scale as well as changes in architecture, training objectives, data strategies and methods for incorporating external information.
