Video codecs are the software that compresses video to a small fraction of its size, so that it can be sent or stored, and decompresses it again for playback. The word codec joins "coder" and "decoder". Audio codecs do the same for sound. Every stream, video call, film file and phone recording depends on one.
Why is compression needed? Raw video is enormous. One second of uncompressed 1080p video, at 30 frames a second, is about 190 megabytes. A streaming service sends the same second in well under one megabyte. Without codecs, no phone network could carry video at all.
Here is how it works in general. Most frames in a video look almost the same as the frame before. So the encoder stores a full picture only now and then. This is called a keyframe. For the frames in between, it stores only what moved or changed, for example "this block of pixels moved ten pixels to the left". It also drops fine detail that the human eye does not notice. This is called lossy compression, because the dropped detail cannot be brought back. On the viewer's device, the decoder rebuilds each frame from the keyframe and the changes. Phones and TVs have a chip that does this decoding quickly, without draining the battery.
Codecs have names. H.264, also called AVC, plays on almost every device. Newer codecs such as H.265 (HEVC) and AV1 need roughly a third to half less data for the same picture. But they take more computing to encode, and older devices cannot play them. Some codecs charge a licence fee for each device, and others are free.
Take, for example, a streaming service. Its encoder compresses each episode several times, into copies at different qualities, using the newest codec that each viewer's phone can play.
Consider a cinema. A digital cinema server receives each film as a compressed file and decodes it frame by frame onto the big screen, at much higher quality than a phone stream.
Say a film is mixed in surround sound. The immersive audio and HDR formats it uses depend on audio codecs that pack many channels of sound into one stream.
How is a codec different from a file format such as MP4? An MP4 file is a container: a box that holds the compressed video, the sound and the subtitles together. The codec is what compressed the video inside the box. A codec is also different from adaptive bitrate streaming, which chooses which compressed copy to send next.
Appears in
- Digital cinema server↗ Ticketing and venue bookingA digital cinema server stores each film compressed with a codec, and decodes it to play on the screen.
- Video encoding and transcoding↗ Video streamingA video encoder uses a codec such as H.264 or AV1 to shrink each video before streaming.
- Immersive audio and HDR formats↗ Film production and VFXImmersive audio and HDR formats are carried inside video files compressed by a codec, which every device must decode.