Speech recognition is the technology of converting spoken language contained in an audio signal into a machine-readable representation, most commonly written text. It is also called automatic speech recognition, or ASR, and sometimes speech-to-text. An ASR system receives an acoustic waveform or derived audio features and estimates the linguistic content spoken by one or more speakers.
The task involves more than simply matching sounds to letters because speech varies with accent, pronunciation, speaking rate, background noise, microphone characteristics, vocabulary, and context. Modern systems commonly use deep neural networks and large-scale training data, with architectures based on recurrent networks, connectionist temporal classification, encoder-decoder models, transformers, conformers, or related approaches. Output can be plain text, timestamped words, phonetic representations, or richer structured transcripts. Speech recognition is distinct from speech synthesis, which generates audio from text or another representation.
The speech-recognition pipeline historically involved several conceptual components. An acoustic model represented relationships between audio features and phonetic or linguistic units, a pronunciation lexicon connected words to possible pronunciations, and a language model estimated the plausibility of word sequences. A decoder combined these sources to find a likely transcription. Modern end-to-end systems can integrate many of these functions into a single neural architecture trained directly to map audio to text or intermediate representations.
Audio is sampled at a defined rate and represented digitally as a sequence of amplitude values. Systems may transform the waveform into spectral features such as log-Mel representations, although some modern models operate more directly on waveforms or learned audio representations. The model then processes temporal information and generates or decodes a sequence of tokens corresponding to the predicted transcription.
Speech recognition is difficult because speech contains acoustic and linguistic ambiguity. Different words can sound identical or very similar, and a single phoneme can have different acoustic realisations depending on surrounding sounds and the speaker. Coarticulation means that pronunciation is influenced by neighbouring sounds, while prosody conveys information through timing, stress, pitch, and intonation. Accents and dialects can change pronunciation patterns and vocabulary. Background noise, reverberation, overlapping speakers, music, compression, and poor microphones can further degrade the signal.
Language models help resolve acoustic ambiguity by considering which word sequences are plausible in context. A phrase that is acoustically ambiguous may become clear when surrounding words are considered. Domain vocabulary also matters: a medical transcription system may need terminology that is uncommon in general speech data, while a technical support system may need product names and specialised abbreviations.
Modern ASR models can be trained using supervised speech datasets containing audio and corresponding transcripts, as well as semi-supervised or self-supervised methods that learn useful acoustic representations from large quantities of unlabelled audio. Self-supervised audio models can learn general representations by predicting masked or transformed portions of an audio sequence and can subsequently be adapted to speech-recognition tasks. Fine-tuning on domain-specific recordings can improve performance for particular accents, vocabulary, microphones, or environments.
Data augmentation can simulate noise, reverberation, speed changes, or other acoustic conditions. Speaker diversity is important because a system trained mostly on one population may perform differently for others. Training-data transcription quality also matters: inaccurate timestamps, missing words, inconsistent punctuation, and incorrect speaker labels can affect downstream performance.
Speech recognition can produce more than raw text. Systems can provide word-level or phoneme-level timestamps, confidence estimates, punctuation, capitalisation, language identification, speaker diarisation, and formatting. Speaker diarisation determines which portions of an audio recording correspond to different speakers, although it does not necessarily identify who those speakers are without additional information. Voice activity detection identifies regions containing speech. Language identification estimates the spoken language. Some systems can handle multilingual speech, code-switching, or multiple languages within one recording.
Streaming ASR processes speech incrementally and provides partial transcripts with low latency, while offline transcription can use the entire recording and potentially achieve better accuracy because future context is available. The choice between streaming and offline recognition depends on latency requirements, computational resources, network conditions, and the application.
Speech recognition has applications in accessibility, dictation, meeting transcription, call-centre analysis, voice search, subtitles, journalism, healthcare documentation, automotive interfaces, customer service, and software control. Accessibility applications can allow users to interact with computers without typing, while automated captioning can make spoken content available as text. Call centres can transcribe conversations for quality assurance or analysis, subject to applicable privacy and governance requirements. Voice assistants combine ASR with natural language processing and other components to interpret spoken commands and generate responses. In healthcare or legal settings, transcription errors can have significant consequences, so generated transcripts may require review. In noisy industrial environments, specialised microphones and domain-specific acoustic models can be as important as the underlying neural architecture.
ASR performance is commonly evaluated using word error rate, or WER, which compares the recognised word sequence with a reference transcription using substitutions, deletions, and insertions. Character error rate can be useful for languages or applications where word segmentation behaves differently. Aggregate accuracy can conceal significant differences across accents, dialects, noise conditions, languages, speakers, and vocabulary. A system may have good average WER but perform poorly for a particular subgroup or specialist terminology.
Recognition errors can also propagate into downstream NLP systems, potentially changing the meaning of a command or transcript. Privacy is a major consideration because speech recordings can contain personal information, sensitive conversations, biometric characteristics, and other data. Secure processing, appropriate retention policies, access controls, and informed handling of recordings are therefore important when deploying ASR systems.
Speech recognition should not be confused with speaker recognition, speech synthesis, voice cloning, or natural language processing. ASR identifies linguistic content from speech; speaker recognition attempts to determine or verify who is speaking; speech synthesis generates speech audio; voice conversion or cloning transforms or reproduces vocal characteristics; NLP analyses the linguistic content after it has been represented as text or another language representation. A complete voice assistant can combine all of these technologies with dialogue management and external tools.
Speech recognition also does not necessarily determine the speaker's intentions, emotions, truthfulness, or meaning beyond what can be inferred from the spoken signal and contextual processing. Its core task is transcription or linguistic decoding of speech, although modern systems can produce additional metadata and can serve as one component in broader multimodal and conversational systems.
