Natural language processing, or NLP, is the field of artificial intelligence and computing concerned with enabling computers to process, analyse, understand, transform, and generate human language. "Natural language" refers to languages used by people, such as English, Spanish, Arabic, Chinese, Japanese, and many others, distinguishing them from formal programming or mathematical languages. NLP encompasses tasks involving written text, and in broader usage can also interact with spoken language when combined with speech technologies.
Major NLP tasks include tokenisation, part-of-speech tagging, syntactic parsing, named-entity recognition, information extraction, sentiment analysis, text classification, question answering, summarisation, semantic similarity, machine translation, dialogue systems, information retrieval, language generation, and language modelling. NLP combines ideas from linguistics, computer science, statistics, information retrieval, machine learning, and increasingly deep learning.
Language is difficult for computers because meaning depends on syntax, context, ambiguity, world knowledge, discourse, pragmatics, cultural conventions, and the relationship between symbols and real-world entities. The word "bank" can refer to a financial institution or the side of a river, and the intended meaning depends on context. Pronouns can require discourse-level information to resolve, while sarcasm can produce wording whose literal interpretation conflicts with the speaker's intended meaning.
NLP systems must also deal with spelling variation, slang, abbreviations, dialects, multilinguality, code-switching, domain-specific terminology, punctuation differences, and noisy user-generated text. Languages differ in morphology, syntax, writing systems, word order, and grammatical features. Consequently, an NLP system trained for one language or domain may perform poorly in another. The field therefore involves both general computational methods and language-specific linguistic considerations.
Early NLP relied heavily on symbolic rules, grammars, dictionaries, hand-built semantic representations, and statistical models. Rule-based systems could encode linguistic patterns explicitly but were often expensive to construct and difficult to maintain at broad scale. Statistical NLP introduced probabilistic models that learned patterns from corpora, including n-gram language models, hidden Markov models, conditional random fields, and probabilistic parsers. Machine learning enabled systems to learn classification and sequence-labelling behaviour from examples rather than requiring every rule to be written manually.
Neural networks subsequently became dominant in many NLP tasks, first through recurrent and convolutional architectures and later through transformer-based models. Large-scale pretrained language models transformed the field by learning general-purpose language representations from extensive datasets and adapting them to many downstream tasks through fine-tuning, prompting, retrieval, or related methods.
NLP commonly begins by representing language in a form suitable for computation. Tokenisation divides text into tokens, although modern systems often use subword tokenisation rather than conventional word splitting. Normalisation can address case, punctuation, Unicode variation, spelling, or other properties depending on the task. Traditional systems used features such as word frequencies, n-grams, grammatical tags, and manually engineered linguistic properties.
Modern neural systems commonly represent tokens as vectors called embeddings, where relationships among words or tokens are represented geometrically in learned vector spaces. Contextual models can produce different representations for the same token depending on surrounding text, allowing polysemy and contextual meaning to be represented more effectively than fixed word embeddings. Sentence and document embeddings can support semantic search, clustering, retrieval, recommendation, and similarity comparison. The representation method strongly influences which linguistic relationships an NLP system can learn.
NLP includes both understanding-oriented and generation-oriented tasks. Named-entity recognition identifies mentions of entities such as people, organisations, locations, dates, or products. Part-of-speech tagging assigns grammatical categories, while syntactic parsing identifies relationships between words or phrases. Information extraction converts unstructured text into structured records. Sentiment analysis estimates attitudes or affect expressed in text, although sentiment does not necessarily equal the author's overall opinion or intent.
Summarisation creates shorter representations of documents, while question answering produces answers conditioned on questions and available information. Text classification can assign documents to categories such as topic, spam status, or support-ticket type. Language generation produces text according to prompts or constraints. Translation converts content between languages. These tasks can be combined into pipelines, but modern foundation models can often perform several of them through a common language-processing architecture.
NLP has practical applications across search engines, customer support, accessibility, document processing, cybersecurity, education, healthcare, finance, law, software development, content management, and communications. Search systems use language processing to understand queries and retrieve relevant information. Support systems classify requests, identify intent, retrieve knowledge, and generate responses. Document-processing systems extract names, dates, clauses, figures, and other structured information from contracts, invoices, reports, and forms. Accessibility technologies can use language generation, summarisation, and speech interfaces to assist users.
Software systems can analyse logs and tickets or generate and transform code. However, high-stakes applications require particular care because NLP systems can misinterpret ambiguous language, propagate bias, omit important qualifications, or generate unsupported statements. Domain-specific evaluation is therefore necessary rather than assuming general language competence guarantees reliability in every specialist context.
NLP should be distinguished from related fields. Speech recognition converts spoken audio into text, while NLP processes the linguistic content of text or other symbolic language representations; a voice assistant may use both sequentially. Machine translation is a specialised NLP task focused on converting meaning between languages. Information retrieval finds relevant existing information, whereas language generation creates new textual sequences, although retrieval and generation are increasingly combined.
Computational linguistics studies computational approaches to human language and has strong overlap with NLP, but it has a broader linguistic and scientific orientation in some contexts. Natural-language understanding refers more specifically to systems that derive structured or semantic interpretations from language, while natural-language generation concerns producing language. These boundaries overlap substantially in modern systems, especially with large language models that can perform multiple language-related tasks within a single architecture.
NLP has important limitations because linguistic form does not guarantee reliable understanding of the underlying world. Statistical language models can generate grammatically fluent text without possessing a complete or consistently accurate model of reality. Bias can arise from training data, annotation practices, language imbalance, or deployment context. Low-resource languages may have substantially less training data and tooling than widely represented languages. Ambiguous references, irony, implicit assumptions, domain-specific jargon, and rapidly changing terminology can cause errors.
Privacy is also important because language data can contain personal or confidential information. Evaluation must consider language, domain, task, demographic variation, robustness, and real-world consequences. Contemporary NLP increasingly combines language models with retrieval, structured knowledge, tools, external databases, and verification mechanisms to improve factual grounding. The field therefore encompasses both the computational processing of linguistic signals and the engineering challenges involved in deploying language technology reliably.
