HLS, in depth: the playlist data model, live sliding windows, segment formats and a validator you can run

The architecture in one picture

An HLS service has four moving parts. The encoder produces a ladder of renditions, which are the same content at different resolutions and bit rates. The packager cuts each rendition into segments and writes playlists. The origin serves those files, and a CDN caches them close to viewers. The player drives everything: it fetches playlists, measures throughput, chooses a rendition and requests segments.

Because the protocol is just files, HLS scales like a static website. A CDN cannot coordinate with a player, so the playlist must carry every promise the player relies on.

Two kinds of playlist

A multivariant playlist (older documents call it the master playlist) lists the variants and alternative renditions. It contains no media. Each #EXT-X-STREAM-INF line describes a variant and is followed by the URI of that variant's media playlist. #EXT-X-MEDIA lines declare groups of alternatives, such as audio languages or subtitles, that variants reference by GROUP-ID.

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="aac",NAME="English",LANGUAGE="en",DEFAULT=YES,AUTOSELECT=YES,URI="audio/en/index.m3u8"

#EXT-X-STREAM-INF:BANDWIDTH=3300000,AVERAGE-BANDWIDTH=2500000,CODECS="avc1.640020,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=30.000,AUDIO="aac"
v720/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=6400000,AVERAGE-BANDWIDTH=4800000,CODECS="avc1.640028,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=30.000,AUDIO="aac"
v1080/index.m3u8

Three attributes do most of the work. BANDWIDTH is required, and RFC 8216 defines it as the peak segment bit rate of the variant, including the audio it is paired with. AVERAGE-BANDWIDTH is optional and is what ABR logic usually wants. CODECS lets the player reject a variant before downloading anything, so a device without HEVC support never switches into an HEVC rung and stalls.

A media playlist lists the segments of one rendition, in order, each preceded by #EXTINF with its duration. Players switch variants only at segment boundaries. Switching is seamless only when renditions are aligned, which means the same segment boundaries, the same media sequence numbers and keyframes at every boundary. Nothing in the playlist enforces that.

The tags that carry the contract

Most of these tags are declarations the player cannot verify. If a packager writes EXT-X-INDEPENDENT-SEGMENTS and a segment begins with a non-key frame, playback after a switch shows corruption until the next keyframe, and the bug report will blame the player.

Segments: MPEG-TS or fragmented MP4

HLS originally carried MPEG-2 transport stream (TS) segments, each self-contained, with codec parameters repeated inside. Fragmented MP4 (fMP4) arrived later. An fMP4 rendition has one initialisation section, referenced by EXT-X-MAP, and media segments that hold only moof and mdat boxes. Because fMP4 segments can be CMAF-compatible, one set of files can serve both HLS and DASH, and the common-encryption schemes used by modern DRM operate on this format.

Segment duration is the main tuning knob, and it trades latency against efficiency. Short segments lower latency and switch faster, but need more keyframes, which cost bits, and more requests. Whatever you pick, force keyframes at exact multiples of the segment duration in every rendition. The packager can only cut at keyframes, so an encoder with free-running GOPs produces segments of uneven length, which breaks alignment and can break the target-duration rule.

Live: the sliding window and its timing rules

A live media playlist has no EXT-X-ENDLIST. The packager appends new segments to the end and removes old ones from the start, increasing EXT-X-MEDIA-SEQUENCE by one for each segment removed. A segment's number is therefore permanent: sequence 1203 means the same file in every version of the playlist. Packagers break this invariant more often than any other.

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:1201
#EXT-X-DISCONTINUITY-SEQUENCE:3
#EXT-X-MAP:URI="init.mp4"
#EXT-X-PROGRAM-DATE-TIME:2026-10-02T12:00:00.000Z
#EXTINF:6.000,
seg_1201.m4s
#EXTINF:6.000,
seg_1202.m4s
#EXTINF:6.000,
seg_1203.m4s
#EXTINF:6.000,
seg_1204.m4s
#EXTINF:6.000,
seg_1205.m4s

RFC 8216 puts rules on both sides of this window. On the server side, the rules are:

• Every EXTINF rounded to the nearest integer must be no greater than EXT-X-TARGETDURATION. A 6.4-second segment rounds to 6 and is legal under a target of 6. A 6.6-second segment rounds to 7 and is not.
• Segments must not be removed if that would leave a live playlist shorter than three target durations.
• Segments that have been removed must remain downloadable for a while, roughly the duration of the playlist at the time of removal, because a player holding an older copy of the playlist may still request them.

On the player side, the rules are:

• Do not start a live stream at a segment that begins less than three target durations from the end of the playlist.
• After loading a playlist that has changed, wait at least one target duration before reloading. If a reload returns an unchanged playlist, retry after half a target duration.

These rules exist because the CDN layer is eventually consistent. Three target durations of headroom means a stale playlist still lists segments that exist and a fresh one has not yet dropped what the player is about to fetch.

Worked example: where a live player starts and why latency is what it is

Take the playlist above. The target duration is 6 seconds, five 6-second segments are listed, and the newest is seg_1205. The window covers 30 seconds of media.

The player must start at least 3 × 6 = 18 seconds from the end. Counting back from the end of 1205, segments 1205, 1204 and 1203 cover 18 seconds, so the latest legal starting segment is 1203. Playback therefore begins about 18 seconds behind the newest media the playlist lists.

Now add what happened before the player saw that playlist. Segment 1205 could only be listed once it was complete, encoded, packaged and uploaded, which adds a second or two after its last frame. The player also sees each new playlist late, by up to a reload interval plus any CDN cache TTL, or about 3 seconds on average with 6-second reloads. That gives 18 + 2 + 3, about 23 seconds glass to glass on average and closer to 26 at worst, which is why classic HLS with 6-second segments sits in the 20 to 30 second range.

The arithmetic also shows where the levers are. Halving the segment duration to 3 seconds (with target duration 3) cuts the hold-back to 9 seconds and the average staleness to 1.5, which lands near 12 or 13 seconds. To get much lower you need LL-HLS partial segments, because the three-target-duration rule scales with the segment size. Cache media playlists for at most half a target duration and segments for a long time.

Discontinuities, ad breaks and wall-clock time

When the stream switches from programme to an inserted ad, the timestamps, encoder settings or even codec profile change. The packager marks the boundary with #EXT-X-DISCONTINUITY so the player resets its timestamp mapping instead of trying to play the ad 40 minutes into the future. When a discontinuity tag slides out of the window, EXT-X-DISCONTINUITY-SEQUENCE increments. Without it, a player that joins late cannot tell which discontinuity domain a segment belongs to, and audio and video renditions packaged separately can drift apart.

EXT-X-PROGRAM-DATE-TIME ties a segment to wall-clock time. It lets a player line up timed metadata and measure latency. Write a fresh value on the first segment after each discontinuity, because the wall-clock mapping does not carry across the break, and keep the encoder clock on NTP.

Encryption: AES-128 and SAMPLE-AES

#EXT-X-KEY:METHOD=AES-128,URI="https://keys.example/k1" encrypts each whole segment with AES-128 in CBC mode with PKCS7 padding. If the tag carries no IV attribute and uses the default identity key format, the IV is the segment's media sequence number as a 128-bit big-endian integer. That is convenient, but it ties decryption to correct sequence numbering, so a packager that renumbers after a restart produces segments that fail to decrypt. SAMPLE-AES encrypts only the media samples and leaves container headers clear. Commercial DRM uses that family together with a key format attribute that names the DRM system.

AES-128 with a key URI gives you only access control. Anyone who can fetch the key can decrypt the content, so protect the key endpoint with short-lived tokens and treat it as authentication rather than DRM. See DRM packaging for how one fMP4 set is encrypted once and served to several systems.

A playlist validator you can run in CI

Most outages trace back to a handful of rules, and those rules are cheap to check. The validator below parses a media playlist and checks the target-duration rule and the three-target-duration window. If given the previous version of the same playlist, it also checks that sequence numbers never go backwards and that a sequence number never changes its URI.

import math, re

def parse_media_playlist(text):
    lines = [l.strip() for l in text.splitlines() if l.strip()]
    if not lines or lines[0] != "#EXTM3U":
        raise ValueError("first line must be #EXTM3U")
    pl = {"target": None, "seq": 0, "segments": [], "endlist": False}
    dur = None
    for l in lines[1:]:
        if l.startswith("#EXT-X-TARGETDURATION:"):
            pl["target"] = int(l.split(":", 1)[1])
        elif l.startswith("#EXT-X-MEDIA-SEQUENCE:"):
            pl["seq"] = int(l.split(":", 1)[1])
        elif l.startswith("#EXTINF:"):
            dur = float(l.split(":", 1)[1].split(",", 1)[0])
        elif l == "#EXT-X-ENDLIST":
            pl["endlist"] = True
        elif not l.startswith("#"):
            if dur is None:
                raise ValueError(f"URI without EXTINF: {l}")
            pl["segments"].append((pl["seq"] + len(pl["segments"]), dur, l))
            dur = None
    return pl

def check(pl, previous=None):
    errs = []
    if pl["target"] is None:
        errs.append("missing EXT-X-TARGETDURATION")
    for n, d, uri in pl["segments"]:
        if pl["target"] is not None and round(d) > pl["target"]:
            errs.append(f"seg {n} lasts {d}s, rounds above target {pl['target']}s")
    total = sum(d for _, d, _ in pl["segments"])
    if not pl["endlist"] and pl["seq"] > 0 and pl["target"] and total < 3 * pl["target"]:
        errs.append(f"removed segments left {total:.1f}s, under 3 x target")
    if previous:                          # compare with the last version we saw
        old = {n: uri for n, _, uri in previous["segments"]}
        for n, _, uri in pl["segments"]:
            if n in old and old[n] != uri:
                errs.append(f"media sequence {n} changed URI: {old[n]} -> {uri}")
        if pl["seq"] < previous["seq"]:
            errs.append("EXT-X-MEDIA-SEQUENCE went backwards")
    return errs

Run it in CI against a short test encode, and in production poll each live rendition from outside your CDN once per target duration, keeping the previous parse so sequence regressions surface within seconds.

Failure modes in production

• Target duration violated. A late keyframe stretches one segment to 7.2 seconds under a target of 6. Strict players reject the playlist and lenient ones mistime reloads. Fix the GOP, not the target.
• Sequence reset after a packager restart. Media sequence starts again at 0, players see old numbers and either stall or replay content, and AES-128 IVs no longer match. Persist the counter, or derive it from the wall clock.
• Unaligned renditions. Switching causes a skip or repeated frames because 720p segment 1203 does not cover the same time range as 1080p segment 1203.
• Playlist over-cached at the CDN. A long TTL on live playlists freezes viewers on a stale window until segments expire, and then they stall. Cache playlists briefly and segments long.
• BANDWIDTH understated. Declaring the average as the peak makes players pick a rung the network cannot sustain during complex scenes, which causes rebuffering on the hardest content. ABR algorithms can only be as good as these numbers.

Trade-offs

What to do next

1. Pull one live media playlist from your service twice, one target duration apart, and run the validator on both versions.
2. Check every rendition's GOP: keyframes must sit exactly at segment boundaries in all of them.
3. Make sure each variant has an accurate peak BANDWIDTH, an AVERAGE-BANDWIDTH and a correct CODECS string.
4. Set CDN TTLs to at most half a target duration for media playlists and long for segments, and keep removed segments on the origin for at least one window length.
5. Persist the media sequence and discontinuity sequence counters across packager restarts, and test the restart path.
6. Work out your latency budget with the worked example. If it is too high, shorten segments or read the live streaming pipeline and the LL-HLS article next.