MP3 vs AAC vs Opus for Streaming, in depth: compatibility, quality per bit, containers, gapless playback and a decision guide

The three questions behind the choice

Every format decision for streaming reduces to three questions. First, who can decode it: browsers, phones, smart speakers and car head units do not all support the same codecs, and a stream nobody can play is worthless. Second, what quality each bit buys: bandwidth is a cost to you and, on mobile data, to your listener. Third, what your pipeline can package: the codec must fit into the containers and manifests your players expect. Keep these three in mind; the rest of the article is detail on each.

The codecs side by side

Opus is a hybrid of a speech coder (SILK) and a transform coder (CELT) that switches between them as the content and bitrate demand; the Opus architecture article explains the modes. AAC is a pure transform coder; its HE profiles reconstruct high frequencies with spectral band replication (SBR) and, in v2, encode stereo as mono plus parametric stereo (PS) data. MP3 is the oldest and least efficient design, but its decoders are in everything.

The pipeline at a glance

The shape is the same for every serious service. Start from a lossless master, normalise loudness so renditions sound equally loud, and resample once to the target rate. Encode every rendition from that prepared file; transcoding from one lossy format to another stacks two sets of artefacts and is the most common quality mistake. Package each rendition in the containers its players expect, and publish manifests that describe every rendition accurately so players choose correctly.

# Always encode every rendition from the lossless master, never from another lossy file.
ffmpeg -i master.wav -af loudnorm=I=-16:TP=-1.5:LRA=11 -ar 48000 prepared.wav

# AAC-LC, 128 kb/s stereo, in fragmented MP4 suitable for HLS or DASH
ffmpeg -i prepared.wav -c:a aac -b:a 128k -movflags +frag_keyframe+empty_moov aac128.mp4

# Opus, 96 kb/s stereo, in WebM for browsers and in MP4 for CMAF packaging
ffmpeg -i prepared.wav -c:a libopus -b:a 96k -vbr on opus96.webm
ffmpeg -i prepared.wav -c:a libopus -b:a 96k -vbr on opus96.mp4

# MP3, 128 kb/s, for progressive download and legacy devices (LAME writes the gapless info tag)
ffmpeg -i prepared.wav -ar 44100 -c:a libmp3lame -b:a 128k legacy128.mp3

Encoder choice matters as much as codec choice. LAME is the reference-quality MP3 encoder. For AAC, FFmpeg's native aac encoder is acceptable at 128 kb/s and above, while Fraunhofer FDK AAC (libfdk_aac, not included in standard FFmpeg builds for licensing reasons) is the usual choice for HE-AAC. For Opus, libopus is the reference encoder; the Opus parameters article explains its bitrate, complexity and frame-size settings.

Quality per bit, honestly

At about 128 kb/s stereo, all three codecs with good encoders are transparent or close to it for most listeners on most music. The differences appear as the bitrate falls. Below roughly 96 kb/s, MP3 develops audible pre-echo and a smeared, watery top end. AAC-LC holds up to around 96 kb/s, and HE-AAC keeps a full-sounding signal at 48 to 64 kb/s by synthesising treble rather than coding it. Opus is generally regarded as the strongest of the three in the 32 to 96 kb/s range, and it handles speech especially well at low rates.

Treat such rules of thumb as starting points, not guarantees. Quality depends on content (solo piano and castanets are hard; compressed pop is easy), encoder version and settings. Before fixing your ladder, listen to your own catalogue in blind comparisons, and if you need numbers at scale, use an objective model such as ViSQOL as a screening tool and confirm its findings by ear.

Compatibility: who decodes what

Safari is the reason Opus cannot be your only format. Safari's Opus support arrived in stages, first for WebM Opus in the audio element, with gaps in other paths such as Web Audio decoding, and behaviour has differed by version and platform. For HLS, Apple's authoring specification for Apple devices lists AAC variants, FLAC, Apple Lossless and Dolby formats for fragmented MP4, but not Opus; an answer on Apple's developer forums states that Opus in HLS works only from iOS 17 and only in MP4. The safe reading is: Opus can be an optimisation for capable clients, but every HLS stream should also carry an AAC rendition.

Do not guess support from the user agent string. Ask the browser, as below, and fall back in order of preference.

// Pick a format the device can actually decode before choosing a URL.
function pickRendition() {
  const a = document.createElement("audio");
  // iPhone Safari has no window.MediaSource; native HLS gives it the AAC renditions.
  if (a.canPlayType("application/vnd.apple.mpegurl")) return "hls-native";
  const ms = window.MediaSource;
  const can = (t) => ms && MediaSource.isTypeSupported(t);
  if (can('audio/mp4; codecs="opus"')) return "opus-cmaf";     // MSE path, Opus in fMP4
  if (can('audio/webm; codecs="opus"')) return "opus-webm";
  if (can('audio/mp4; codecs="mp4a.40.2"')) return "aac-cmaf";
  if (a.canPlayType('audio/mp4; codecs="mp4a.40.2"')) return "aac-progressive";
  if (a.canPlayType("audio/mpeg")) return "mp3-progressive";   // last resort
  return null;
}

Containers and manifests

Each codec travels in a container. MP3 is usually served as a plain .mp3 file for progressive download or as an endless stream from Icecast-style servers. AAC appears as raw ADTS frames for internet radio, in MPEG-TS segments for older HLS, and in fragmented MP4 (CMAF) for modern HLS and DASH. Opus lives in Ogg (RFC 7845 defines the mapping), WebM, and MP4, which lets it share a CMAF pipeline with AAC for DASH and capable HLS clients.

Manifests must describe codecs precisely, using the CODECS attribute in HLS or codecs in DASH. AAC-LC is mp4a.40.2, HE-AAC v1 is mp4a.40.5 and HE-AAC v2 is mp4a.40.29; Opus in MP4 is identified as opus. A wrong string makes players either reject a playable stream or pick one they cannot decode.

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-STREAM-INF:BANDWIDTH=140000,CODECS="mp4a.40.2"
aac128/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=105000,CODECS="mp4a.40.2"
aac96/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=56000,CODECS="mp4a.40.5"
heaac48/index.m3u8

For live streams, the codec contributes only tens of milliseconds of delay, while segment-based delivery adds seconds. If you need conversational latency, segments are the wrong tool; see audio streaming protocols compared for WebRTC and low-latency HLS.

Priming, padding and gapless playback

Every lossy encoder produces some silence at the start, called priming or encoder delay, and pads the end to fill the last frame. Albums with continuous music, such as live recordings or DJ mixes, then click or gap between tracks unless the player knows exactly how many samples to drop.

Each format records this differently. MP3 has no field for it in the standard; LAME writes the delay and padding into its info tag, and players that read the tag play gaplessly while others do not. AAC in MP4 uses edit lists or iTunes-style metadata; Apple's encoders typically add 2112 priming samples. Opus puts a pre-skip value in the Ogg header, commonly 312 samples at 48 kHz with libopus defaults, and decoders are required to discard it. These figures are typical encoder defaults, not constants: always read them from the file rather than hard-coding them.

Frame sizes and segment boundaries

Segmented streaming adds a subtle constraint. An AAC-LC frame at 48 kHz lasts 1024 / 48000 s, about 21.33 ms, so a 6-second segment holds 281.25 frames. Segments therefore cannot end exactly at 6.000 s, and durations drift by a fraction of a frame per segment. Players tolerate this, but when audio and video segments must line up, or ad insertion splices at exact times, it causes small gaps and timestamp jumps.

Two remedies exist. Choose segment durations that are whole numbers of frames, such as 375 AAC frames at 48 kHz, which is exactly 8.0 s, or accept slightly irregular durations and let the packager write precise values. Opus at 20 ms frames divides any whole number of seconds cleanly (6 s is 300 frames), which is one quiet operational advantage. MP3 at 44.1 kHz has frames of about 26.12 ms, which divide nothing neatly.

Worked example: a ladder and its cost

Suppose a podcast and music service with ten million listening hours a month, a third of them on Apple devices. A sensible ladder is AAC-LC at 128 and 96 kb/s, HE-AAC at 48 kb/s for poor mobile connections, Opus at 96 and 64 kb/s for browsers and Android, and one MP3 at 128 kb/s for progressive download and legacy devices. Speech-only podcasts can use mono renditions at half these rates.

Bandwidth arithmetic is simple. One hour at 128 kb/s is 128,000 × 3600 / 8 bytes, about 57.6 MB; at 64 kb/s, about 28.8 MB. If the two thirds of hours that can use Opus move from AAC 128 to Opus 96, each hour drops from 57.6 to 43.2 MB, saving about 96 TB a month across 6.7 million hours. Whether that justifies a second codec depends on your CDN price and on the extra storage, encoding and testing work, which is why many services adopt Opus only for their largest platforms.

Failure modes

• Silence on some devices. A player picked an Opus rendition it cannot decode. Fix the CODECS strings and keep AAC available.
• Generation loss. Renditions made from an MP3 or AAC mezzanine. Always encode from the lossless master.
• Loudness jumps. Switching renditions changes the level. Normalise once, before encoding.
• Clicks between tracks. Priming not signalled or not honoured. Test gapless albums on every target player.
• Phasey stereo. HE-AAC v2 used for wide music where parametric stereo collapses the image. Reserve v2 for the lowest rungs.
• Wasted bits. Stereo renditions for mono speech, or HE-AAC at high rates, where AAC-LC is more efficient.

A decision guide

What to do next

1. List your playback targets and test which codecs and containers each actually decodes.
2. Make sure every rendition is encoded from a lossless, loudness-normalised master.
3. Keep an AAC rendition in every HLS stream; add Opus only where clients advertise support.
4. Validate CODECS strings in every manifest and test gapless playback on a continuous album.
5. Blind-listen to your hardest content at each rung before fixing the ladder.
6. Compute bandwidth per listening hour for each rung and compare it with the cost of maintaining another codec.