What is a Video Codec?

Technology — James William Steven Parker

A video codec is a system of algorithms and specifications used to encode and decode digital video. The word codec is formed from "coder" or "compressor" and "decoder" or "decompressor". During encoding, the codec converts source video into a coded representation suitable for storage or transmission, usually reducing its size through compression. During decoding, compatible software or hardware reconstructs a sequence of images from that coded representation. Examples include H.264, H.265 and AV1.

A codec is distinct from a container format: a codec defines how the video is represented, whereas a container such as MP4 or WebM can package video together with audio, subtitles and metadata. Codec design is fundamentally a trade-off between compression efficiency, visual quality, computational complexity, latency, compatibility and implementation cost.

Video codecs exploit both spatial and temporal redundancy. Adjacent pixels within a frame often contain related information, while successive frames commonly differ only in moving regions or scene changes. Intra-frame coding compresses a picture using information within that picture, while inter-frame coding can predict a current picture from previously or subsequently coded pictures. Modern codecs use techniques such as block or partition-based prediction, motion estimation, motion compensation, transforms, quantisation, entropy coding, in-loop filtering and sophisticated reference-picture management. The encoder searches for efficient ways to represent the video, whereas the decoder normally performs a more constrained reconstruction process. This asymmetry is why encoding can be substantially more computationally expensive than decoding, particularly at high quality or with advanced codec settings.

Lossy video codecs normally reduce file size by discarding information that the encoder estimates will have limited perceptual importance. Transform coefficients can be quantised so that fine detail and subtle variations require fewer bits. The quantisation process is a principal source of irreversible loss in conventional lossy encoding. Excessive compression can produce blocking, ringing, mosquito noise, texture smearing and loss of fine detail.

Lossless modes or codecs instead preserve the source information exactly, but their compression ratios are generally much lower for complex natural video. Codec quality is therefore not simply a property of the codec name. Encoder implementation, bitrate, rate-control method, source characteristics, resolution, frame rate, bit depth and codec settings can produce substantially different results even when the same codec is used.

Inter-frame codecs use concepts such as I-frames, P-frames and B-frames, although terminology varies among standards. An intra-coded frame can be decoded without relying on other pictures, while predictive frames reference previously decoded information. Bidirectionally predicted pictures can use reference pictures on both sides in display or coding order. This can greatly reduce bitrate for ordinary motion but introduces dependencies between frames. More reference pictures and sophisticated prediction can improve compression efficiency while increasing memory, latency and computational requirements. Scene changes often require a new intra-coded representation because previous frames no longer provide useful predictions. Codec structures therefore influence seeking, editing, error recovery and streaming latency as well as compression efficiency.

Rate control determines how the encoder allocates bits over time. Constant bitrate attempts to keep the encoded data rate within a relatively stable range, which can be useful for constrained channels. Variable bitrate allows the encoder to use more bits for complex scenes and fewer for simple scenes, often improving quality at a given average bitrate. Constant-quality or quality-targeted approaches attempt to maintain a chosen perceptual or mathematical quality level while allowing bitrate to vary.

Modern encoders may use lookahead, scene analysis and psychovisual optimisation to decide where bits are most valuable. A static scene with little motion can require far fewer bits than rapidly changing foliage, water, smoke or a detailed sports sequence because the latter contains much more information that is difficult to predict efficiently.

Codec support involves more than knowing the codec's name. Profiles and levels can restrict available tools, maximum resolution, frame rate, bitrate or decoding requirements. A device may support a codec but not a particular profile, level, bit depth or chroma format. Hardware acceleration can dramatically affect practical playback performance, especially for high-resolution video. Devices may decode common formats directly in dedicated silicon while relying on general-purpose processors for less common formats, increasing power consumption. Encoding and decoding support can also differ: a phone may efficiently decode a codec but be unable to encode it in hardware. Compatibility therefore depends on the codec specification, implementation, container, operating system, hardware and actual stream parameters.

Codecs are important in almost every modern video workflow. Cameras use codecs to record footage, editing applications decode and encode media, streaming services create multiple encoded versions, video conferencing systems compress real-time camera feeds, and broadcast systems use standardised coding schemes for transmission. Production environments often use less aggressive codecs or intraframe-oriented formats because editing and repeated processing benefit from easier decoding and higher retained quality. Distribution systems favour codecs that achieve good quality at relatively low bitrate. Real-time communication has additional constraints because encoding and decoding must occur with very low delay. The best codec choice therefore depends on whether the priority is editing, archiving, live transmission, web distribution, device compatibility or another purpose.

The history of video codecs reflects the increasing demand to transmit and store higher-quality video efficiently. Earlier digital standards provided substantial improvements over raw video, while successive generations introduced more sophisticated prediction, transforms, entropy coding and filtering. H.264 became extremely widespread because it offered strong compression efficiency and broad hardware and software support. H.265 improved compression efficiency further, particularly for high-resolution material, while AV1 was developed as a modern, royalty-free video coding standard with an emphasis on efficient internet distribution.

Newer codecs and future standards continue to pursue better compression, lower complexity, higher dynamic range and support for emerging applications. A codec is therefore best understood as a precisely defined language for representing video data, implemented through encoders and decoders that transform between source imagery and compressed digital streams.