Why architecture matters here
The reason to care starts with storage math and ends with agility. A single title, encoded into a ladder of six to eight bitrates, is already several copies of the video. Multiply that by the packaging dimensions — HLS and DASH manifests, TS versus fMP4 versus CMAF containers, clear plus three DRM systems — and a naive static pipeline stores ten to twenty times the mezzanine size per title. Across a catalog of tens of thousands of titles that is a storage bill measured in petabytes, most of it redundant re-wrappings of the exact same encoded bytes. JIT packaging stores the encoded bytes once and manufactures the wrappers on demand, so origin storage collapses back toward the size of the mezzanine itself.
The deeper win is catalog agility, and it is usually worth more than the storage. In a static world, every packaging decision is baked into files: if you want to support a new device that needs a slightly different manifest, or add a fourth DRM system, or change your segment duration, you must re-package — potentially re-package the entire library, a job measured in days of compute and a migration risk on every title. In a JIT world those are configuration changes to the packager. A new device profile is a new output template; a new DRM system is a new key-provider integration; a manifest tweak takes effect on the next cache miss. The catalog stops being a frozen set of artifacts and becomes a live function of the packager's configuration.
The trade-off is that you have moved packaging work onto the request path, and packaging is not free. Building a manifest, rewrapping a segment's container, and applying encryption all cost CPU, and now that cost is paid — at least on cache misses — while a viewer waits for playback to start or for the next segment to arrive. The entire viability of JIT rests on the observation that video traffic is enormously cacheable: the same popular titles, in the same handful of formats, are requested over and over, so the packager does the real work once per (segment, format) and the CDN serves the cached result millions of times. The architecture is a bet that cache hit ratio will be high enough that per-request packaging cost is amortized to near zero.
That bet reshapes how you reason about capacity. The questions become 'what is my edge cache hit ratio for packaged output?' and 'how much packager compute do I need to absorb the cache-miss rate, including the cold-start burst when a new title goes live?' rather than 'how much storage do I need for every format of every title?' Those are cache-and-compute questions about a delivery tier, and answering them well is what separates a JIT system that feels instant from one that stutters at the start of every stream. A miss on a cold title is a packaging job on the critical path; a hit is a static-file serve — and the ratio between them is the whole ballgame.
There is also a correctness dimension that pure storage math hides. Because the packager builds output deterministically from one mezzanine, every device sees renditions derived from the identical encode, so bitrate-ladder alignment, timestamps, and key rotation stay consistent across formats by construction. In a static pipeline those artifacts are produced by separate packaging runs that can drift, and the mismatches surface as adaptive-bitrate glitches or DRM license failures on one platform but not another. Manufacturing every format from a single source is not just cheaper; it is a consistency guarantee that is hard to maintain when each format is a frozen artifact with its own history.
The architecture: every piece explained
Top row: the request path from player to the bytes. A player asks for a manifest or a segment; the request lands on the CDN edge, which either serves a cached packaged response (the common, cheap case) or, on a miss, forwards to the JIT packager at the origin. The packager is the heart of the system: it reads the neutral mezzanine store — the content encoded once as CMAF/fMP4 segments — and produces the exact output the request asked for. Crucially, the mezzanine is format-agnostic: it holds the encoded video and audio in a container and codec the packager can rewrap into anything, so one stored copy backs every delivery format.
Middle row: what the packager actually does on a miss. The manifest builder generates the HLS playlist or DASH MPD tailored to the requesting device — the right bitrate ladder, the right segment references, the right codec strings and DRM signaling. The segment repackage stage rewraps the mezzanine's stored fragments into the container the client wants (for example, MPEG-TS for legacy HLS versus fMP4/CMAF for modern players) without re-encoding the video — it is a container operation, not a transcode, which is why it is fast enough to do inline. The DRM/encryption stage applies Common Encryption (CENC) using keys fetched from a key server, producing the encrypted segment and the license signaling the manifest advertises. The result is written to the edge cache so the next identical request is a hit.
Bottom rows: the two halves of the system and the ops surface. The ingest side is where the expensive, one-time work happens: encode the source once into the mezzanine ladder. The delivery side is where cheap, repeated work happens: package that mezzanine many times, on demand, into whatever formats clients request. The clean separation — encode once, package many — is the defining shape of JIT. The ops strip names the numbers that govern its health: cache hit ratio (the single most important metric, since misses are the only requests that cost real work), packager CPU (the compute headroom that absorbs the miss rate), cold-request latency (how long a miss takes to package, which viewers feel as startup or rebuffer time), and key-server load (the DRM dependency the encryption stage leans on).
A design nuance ties the pieces together: the mezzanine format is chosen precisely to make repackaging cheap. Modern JIT systems standardize on CMAF fragmented MP4 because a single CMAF segment can be referenced by both an HLS playlist and a DASH manifest and encrypted once with CENC for multiple DRM systems — the industry's convergence on 'package once, play everywhere.' The closer the mezzanine is to the delivery format, the less the packager has to do per request, which directly lowers miss latency and packager CPU. So the architecture's efficiency is baked in at ingest by choosing an intermediate that is a rewrap, not a re-encode, away from every output — the packager should never touch the codec, only the wrapper and the encryption.
End-to-end flow
Follow a viewer pressing play on a title that is already popular. The player requests the HLS manifest; the request hits the CDN edge and finds it cached, because thousands of other viewers on the same platform requested the same manifest minutes ago. The edge returns it in milliseconds — no packager involved. The player then requests the first media segment at its chosen bitrate; that too is a cache hit, served as a static file. Playback starts fast, and from the packager's perspective this entire session may cost zero packaging work: every byte came from cache. This is the design working as intended, and for popular content it is the overwhelming majority of traffic.
Now follow the cold case that reveals the machinery: a viewer opens a title that just went live, or requests a rare format nobody has fetched yet. The manifest request misses at the edge and forwards to the JIT packager. The packager reads the mezzanine, the manifest builder assembles the HLS playlist for this device, and the response is returned and cached. The player then requests segment one; that misses too, so the packager reads the corresponding mezzanine fragment, rewraps it into the client's container, fetches the content key from the key server, applies CENC encryption, and returns the packaged encrypted segment — which is now cached at the edge for every subsequent viewer. The first viewer paid the packaging latency; everyone after them gets a hit.
The two paths diverge sharply in cost, and that divergence is the whole performance model. A hit is a static serve — CDN latency, nothing more. A miss is a packaging job on the critical path: read mezzanine, rewrap, encrypt, fetch key. Because rewrapping is a container operation rather than a transcode, and because keys are cached, a warm packager turns a miss around in tens of milliseconds — fast enough that even the first viewer of a new title sees acceptable startup. The system is therefore tuned so that misses are rare (high cache hit ratio) and cheap (rewrap-not-transcode, keys cached), and the combination is what makes packaging-on-demand viable at the scale of live streaming and large catalogs.
The performance character deserves precision. Startup latency for popular content is dominated by CDN round-trips, identical to a static pipeline, because everything is cached. Startup latency for cold content has an added floor equal to the packager's per-segment packaging time plus, on the very first request, a key-server round trip. Steady-state playback of any title — popular or freshly warmed — is all cache hits, because once a viewer walks the bitrate ladder through a title, those segments are packaged and cached for the next viewer. So the only viewers who ever feel packaging latency are the first few on a cold (segment, format) tuple, and even they feel only tens of milliseconds if the packager is warm and provisioned. The architecture concentrates its one real cost onto the leading edge of demand and amortizes it across everyone behind.
Consider the stress case that exposes the sharp edge: a live event or a marketing push that makes a brand-new title go viral in seconds. Every segment of a live stream is cold on first request — it did not exist a moment ago — so a surge of viewers all miss on the same fresh segment simultaneously. Without protection, thousands of concurrent misses for the identical segment stampede the packager, each independently reading the mezzanine, rewrapping, and encrypting the same bytes. The mitigation is request coalescing at the edge: collapse concurrent misses for the same object into a single origin packaging request and fan the one packaged result out to all waiters. With coalescing, a million concurrent viewers of a live segment cause one packaging job, not a million — which is what lets JIT packaging serve live at scale, given enough packager headroom for the genuine live-edge miss rate.