HDR Video Architecture, in depth: PQ and HLG, BT.2020, static and dynamic metadata, encoding, signalling and tone mapping

Four things change, not one

A pipeline has to carry all four differences intact.

The bit depth requirement follows from the range. Spreading 0.005 to 1,000 nits over 8 bits leaves steps big enough to see as bands in skies and gradients. Ten bits gives four times as many codes, and the PQ curve places them where the eye is most sensitive, which is the subject of the next section.

PQ: a transfer function built from the eye

The Perceptual Quantizer, standardised as SMPTE ST 2084, maps a signal in the range 0 to 1 to an absolute luminance between 0 and 10,000 cd/m2 (nits). Its shape was fitted to measurements of the smallest luminance step a viewer can detect at each brightness, so that each code step sits close to that threshold everywhere in the range. The inverse EOTF, which turns luminance into signal, is

Y  = L / 10000                      # normalised luminance
E  = ((c1 + c2 * Y**m1) / (1 + c3 * Y**m1)) ** m2

m1 = 2610/16384      = 0.1593017578125
m2 = 2523/4096 * 128 = 78.84375
c1 = 3424/4096       = 0.8359375
c2 = 2413/4096 * 32  = 18.8515625
c3 = 2392/4096 * 32  = 18.6875

Because PQ is absolute, a code means the same luminance on every compliant display. In 10-bit narrow range, 64 is black and 940 the nominal top.

Two numbers explain most HDR behaviour. SDR's entire range, up to 100 nits, uses only the bottom 51% of the PQ signal, so most code values describe shadows and midtones, where the eye is most sensitive. And everything from 1,000 to 10,000 nits fits in the top 217 codes, because the eye resolves bright steps coarsely.

HLG: compatible by design

Hybrid Log-Gamma, defined in ARIB STD-B67 and included in ITU-R BT.2100, was designed by the BBC and NHK for broadcast, where one signal has to serve both SDR and HDR sets and there is no channel for per-show metadata. HLG describes light relative to the scene rather than absolute display luminance. Its OETF is a square root for the darker part of the range and a logarithm for highlights:

# E: normalised scene linear light, 0..1. Returns signal 0..1.
a = 0.17883277
b = 1 - 4 * a                 # 0.28466892
c = 0.5 - a * math.log(4 * a) # 0.55991073

def hlg_oetf(E):
    if E <= 1 / 12:
        return math.sqrt(3 * E)
    return a * math.log(12 * E - b) + c

The square-root segment resembles a conventional camera curve, so an SDR set showing an HLG signal produces an acceptable picture without understanding it. An HLG-aware display applies a system gamma that depends on its own peak brightness, so a 1,000-nit set and a 400-nit set both render the scene plausibly. The cost is less creative control over highlights, which makes HLG the usual choice for live broadcast and PQ for graded on-demand content.

Colour: a big container and a smaller picture

HDR signals use BT.2020 primaries, a gamut much wider than BT.709. No consumer display covers all of BT.2020, and most content is graded on a monitor that covers DCI-P3, so the picture occupies the P3 region inside a BT.2020 container. The mastering display metadata, covered next, tells the receiver which gamut was actually used.

Two colour details catch pipelines out. First, the YCbCr matrix must be the BT.2020 non-constant-luminance matrix; converting with BT.709 coefficients shifts hues, most visibly in skin and saturated reds. Second, every conversion must run at 10 bits or more from end to end. One 8-bit intermediate, often a filter graph default, reintroduces banding that no later stage can remove.

Metadata: static, dynamic, and where it rides

HDR10 is PQ, BT.2020, 10-bit, plus static metadata that applies to the whole title. SMPTE ST 2086 describes the mastering display: its primaries, white point, and minimum and maximum luminance. Content light level information adds MaxCLL, the brightest single pixel in the title, and MaxFALL, the highest frame-average light level, both in nits. In HEVC these travel in SEI messages; AV1 carries them in metadata OBUs.

Static metadata has an obvious weakness. A dark film with one 1,000-nit explosion has a MaxCLL of 1,000, so a 500-nit TV compresses the whole film's highlights to make room for one shot. Dynamic metadata fixes this by describing each scene or frame. HDR10+ (SMPTE ST 2094-40) carries per-scene luminance distribution data in SEI. Dolby Vision carries its own per-frame metadata, the RPU, generated by Dolby's analysis tools, and is defined in numbered profiles.

Encoding: signal everything, measure the light levels

A conformant HDR10 encode needs four things in the bitstream: 10-bit samples, colour description in the VUI (primaries 9 for BT.2020, transfer 16 for PQ or 18 for HLG, matrix 9 for BT.2020 non-constant luminance), mastering display metadata, and content light levels. With FFmpeg and x265:

ffmpeg -i master_p3_1000nit.mov -map 0:v:0 \
  -c:v libx265 -pix_fmt yuv420p10le -preset slow -crf 18 \
  -color_primaries bt2020 -color_trc smpte2084 -colorspace bt2020nc \
  -x265-params "hdr10=1:hdr10-opt=1:repeat-headers=1:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:range=limited:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1):max-cll=1000,400" \
  hdr10_out.mp4

The master-display string gives chromaticities in units of 0.00002 and luminance in units of 0.0001 nits, so the example describes a P3 display with a D65 white point, 1,000-nit peak and 0.0001-nit black. repeat-headers=1 repeats parameter sets and SEI at every keyframe, which streaming needs because playback can start at any segment. hdr10-opt enables x265's luma and chroma QP adjustments for PQ content; older x265 releases spell these options hdr and hdr-opt.

Do not copy max-cll values from someone else's command; measure them. MaxCLL and MaxFALL come from the decoded, linear-light frames:

import numpy as np

def pq_decode(e):                      # PQ signal 0..1 -> nits, per ST 2084
    ep = np.power(e, 1 / 78.84375)
    y = np.maximum(ep - 0.8359375, 0) / (18.8515625 - 18.6875 * ep)
    return 10000 * np.power(y, 1 / 0.1593017578125)

def light_levels(frames):              # frames: iterable of HxWx3 float RGB, PQ-encoded 0..1
    max_cll, max_fall = 0.0, 0.0
    for rgb in frames:
        nits = pq_decode(rgb).max(axis=2)       # brightest component per pixel
        max_cll = max(max_cll, float(nits.max()))
        max_fall = max(max_fall, float(nits.mean()))
    return round(max_cll), round(max_fall)

Some teams use a high percentile instead of the absolute maximum so a few hot pixels do not set MaxCLL; either way, be consistent.

Packaging and signalling

A player has to learn that a rendition is HDR before it downloads it, or it will fetch segments the device cannot show. In HLS, each EXT-X-STREAM-INF carries VIDEO-RANGE=PQ, HLG or SDR, and Dolby Vision streams add a SUPPLEMENTAL-CODECS attribute alongside the base codec string. In DASH, the colour description is signalled with CICP descriptors on the adaptation set, using scheme urn:mpeg:mpegB:cicp:TransferCharacteristics with value 16 for PQ or 18 for HLG, plus matching primaries and matrix descriptors.

Put SDR and HDR renditions in separate switching sets: adaptive bitrate logic switches freely within a set, and an HDR-to-SDR switch shows as a jump in brightness and colour. Beyond this, packaging is format-neutral; CMAF packaging and HLS vs DASH cover the containers and manifests.

Keep an SDR ladder made from a deliberate grade or reviewed tone map; many devices and browsers still cannot show HDR.

Tone mapping on the receiving end

Almost no display reproduces a master exactly. A 600-nit TV given a 1,000-nit master must compress the highlights. The usual approach, described in ITU-R BT.2390 as an EETF, leaves everything below a knee point unchanged and rolls the range above the knee smoothly into the display's peak. Since BT.2408 places diffuse reference white at 203 nits (PQ 0.58), a knee above that keeps faces, paper and walls at their mastered level and spends the compression on speculars and skies.

A simplified roll-off in PQ space shows the idea; BT.2390's Hermite spline also matches the slope at the knee, which this version does not:

# pq_encode(nits) is the inverse EOTF shown in the PQ section.
def tone_map_pq(e, src_peak_nits, dst_peak_nits, knee_frac=0.75):
    # e: PQ signal of one pixel's luminance. Returns PQ signal for the target display.
    src_max = pq_encode(src_peak_nits)
    dst_max = pq_encode(dst_peak_nits)
    if src_max <= dst_max:
        return e                                   # display can show it all
    knee = max(knee_frac * dst_max, pq_encode(203))  # never compress reference white
    if e <= knee:
        return e                                   # midtones untouched
    t = (e - knee) / (src_max - knee)              # 0..1 across the compressed region
    t = min(t, 1.0)
    # quadratic ease-out: rises steeply from the knee, flattens at the display peak, no hard clip
    return knee + (dst_max - knee) * (1 - (1 - t) ** 2)

Real implementations apply this to a luminance measure and adjust chroma to preserve hue. With static metadata, the display picks one curve per title from MaxCLL and the mastering peak; with dynamic metadata it picks per scene, which is why a dark scene in a Dolby Vision or HDR10+ title can keep its full contrast on a modest TV.

A worked example

A studio delivers a feature graded in P3 on a 1,000-nit monitor. The service wants HDR10, Dolby Vision 8.1 and SDR.

Analysis reads MaxCLL 950 and MaxFALL 180 from the decoded master, and the Dolby tools produce the RPU. The encoder produces one 10-bit HEVC ladder carrying the HDR10 metadata in SEI and the RPU in the same stream, which serves both Dolby Vision and HDR10 devices. A separate SDR ladder is encoded from the studio's trim pass. Quality is checked per rendition with an HDR-aware metric (see VMAF quality metrics for what the standard models do and do not cover), and the ladder shape comes from per-title encoding, noting that HDR content usually needs somewhat more bits than SDR at the same perceived quality.

The manifest marks the two sets VIDEO-RANGE=PQ and VIDEO-RANGE=SDR; the player picks one by display capability and stays in it.

Failure modes

• Washed-out grey picture: PQ content flagged as BT.709, so the display treats PQ codes as gamma. Check the VUI transfer value in the output, not only the encoder flags.
• Dark, crushed, oversaturated picture: the reverse; SDR content flagged as PQ, often after a template change.
• Banding in gradients: an 8-bit step somewhere in the chain, or an aggressive encode of dark scenes. Inspect the pixel format at every filter.
• Metadata stripped: a remux, transcoder or ad-insertion splice that drops SEI. Probe output segments, not just the first one.
• Brightness jumps: mixed HDR and SDR renditions in one switching set, or ads and content mastered at different reference whites.
• Wrong MaxCLL: copied from a template, so TVs tone map too much or clip.

Trade-offs between formats

What to do next

1. Probe a sample of your output segments and confirm primaries 9, transfer 16 or 18 and matrix 9, plus mastering display and light-level metadata in every keyframe.
2. Audit every filter in your transcode graph for 8-bit intermediates.
3. Measure MaxCLL and MaxFALL from decoded frames for each title and record the method.
4. Keep HDR and SDR renditions in separate switching sets, and keep an SDR ladder.
5. Test playback on at least one low-peak HDR display, where tone mapping problems show first.
6. Decide per content type: HLG for live, PQ with static or dynamic metadata for graded on-demand.