Why architecture matters here
The architecture matters because the deadline forbids the obvious fix. On a lossy link the intuitive response to a dropped packet is to resend it, and for almost every other kind of traffic that is exactly right — TCP's reliability is built on retransmission. But a voice frame is only useful in the fraction of a second around when it should play. By the time a retransmission request travels to the sender and the replacement travels back, the playout clock has already passed that frame's slot; the receiver has either concealed the gap or stalled the whole buffer waiting. Retransmission trades a small loss for a large delay, and on a conversation that delay is worse than the loss. FEC exists precisely to recover loss without spending a round trip, which is the only way to recover it before the deadline.
The second forcing function is that concealment alone is not enough. Every serious voice codec has a packet-loss-concealment (PLC) stage that synthesizes plausible audio for a missing frame from the frames around it — extending a vowel, fading a consonant — and for a single isolated lost packet PLC is often inaudible. But PLC is a guess, and its quality degrades fast as gaps grow: two or three consecutive lost frames and the synthesis becomes an obvious robotic smear or a muted dropout. FEC changes the game by turning many lost packets into recovered packets, so the concealer only has to cover the residue FEC could not rebuild. The two layers compose: FEC recovers what it can, PLC hides the rest, and the audible quality is set by how small FEC keeps the residual loss.
The third reason is that the network's loss is neither constant nor random, so a fixed redundancy level is always wrong. Wireless and congested paths lose packets in bursts — a handful in a row when a radio fades or a queue overflows — separated by long clean stretches. A redundancy scheme tuned for the average loss rate will be wasteful during the clean stretches (spending bandwidth to protect packets that were never at risk) and overwhelmed during the bursts (the redundancy covers one lost packet but the burst dropped four). The architecture therefore has to be adaptive, watching the measured loss and adjusting how much redundancy it sends, and sometimes how it spaces that redundancy so a burst does not take out both a packet and its recovery copy at once.
The fourth architectural payoff is that FEC is a knob the sender controls unilaterally, which makes it deployable. Because the redundancy is added by the sender and consumed by the receiver without any negotiation round trip, a call can raise its protection the instant it detects loss climbing, and drop it the instant the path clears, all within the ongoing media stream. In-band FEC in Opus is especially clean here: the recovery data lives inside the normal packets, so there are no extra packets to route, no separate stream to firewall, and the receiver simply decodes the redundant copy if it needs it and ignores it if it does not. This unilateral, in-stream adaptivity is what lets a voice service hold a call together across a network whose quality swings second to second.
A fifth consideration is that FEC interacts tightly with the jitter buffer, and the two must be designed together rather than bolted on separately. The jitter buffer already holds arriving packets briefly to absorb variation in their arrival timing and to reorder packets that came out of sequence; FEC recovery naturally lives at this same stage, because reconstructing a lost frame requires having the neighboring frames that carry its redundancy in hand. A buffer that is too shallow will declare a packet lost and invoke FEC or concealment before the real packet (or its recovery copy) has had a chance to arrive, wasting redundancy on packets that were merely late; a buffer that is too deep recovers more but pushes end-to-end delay past the conversational limit. Tuning the buffer depth and the FEC aggressiveness as one system is what keeps both loss and delay inside the budget a natural conversation allows.
The architecture: every piece explained
Top row: the send path from speech to wire. The audio encoder compresses each twenty-millisecond slice of captured audio into one coded frame — with Opus, a single packet's worth of speech. The FEC encoder is where protection is added: in-band FEC embeds a low-bitrate copy of the previous frame inside the current packet, while a RED-style scheme bundles a whole earlier frame alongside the current one, so each packet carries both 'now' and a backup of 'a moment ago'. The packetizer wraps these into RTP packets with sequence numbers and timestamps and sends them onto the lossy network, which drops, reorders, and delivers packets in bursts rather than the clean stream the sender emitted.
Middle row: the receive path that turns a gapped stream back into continuous audio. The jitter buffer holds arriving packets briefly, reordering those that arrive out of sequence and giving slightly-late packets a chance to show up before their playout slot. The loss detector watches the RTP sequence numbers and flags a gap — sequence 42 arrived, 44 arrived, so 43 is missing. For that gap the FEC decoder tries reconstruction: it pulls the redundant copy of frame 43 that rode inside packet 44 (or the RED backup) and decodes it, filling the hole with real audio. Only when reconstruction is impossible — the redundancy was itself lost, or the loss ran longer than the protection covers — does the PLC fallback synthesize a concealment frame from the surrounding audio.
Bottom row: the control loop and its cost. Adaptive redundancy is the feedback that keeps the whole thing economical: the sender learns the current loss rate (from receiver reports or its own signaling) and raises FEC when loss climbs, lowers it when the path clears. Bandwidth cost is the counterweight the loop must respect — every redundant copy inflates the transmitted bitrate, so protection is not free and over-protecting a clean link wastes the very capacity whose scarcity may be causing loss. The ops strip names the signals that matter: the raw loss rate, the recovery rate (fraction of lost packets FEC rebuilt), the extra bitrate spent on redundancy, the count of frames left to concealment, and the end-to-end delay the buffer and recovery add.
End-to-end flow
Follow twenty milliseconds of speech across a shaky Wi-Fi call. The encoder produces frame 100 for the current slice. The FEC encoder, running in-band FEC because the receiver recently reported two percent loss, embeds a compact copy of frame 99 inside the same packet. The packetizer stamps it RTP sequence 100 and sends it. On the network, packet 99 is dropped by a momentary radio fade, but packet 100 gets through. At the receiver the jitter buffer holds packet 100; the loss detector notices sequence 99 never arrived and that its playout slot is approaching. Instead of concealing, the FEC decoder reaches into packet 100, extracts the redundant copy of frame 99, decodes it at its lower fidelity, and plays it in slot 99. The listener hears continuous speech; the fade was invisible.
Now the loss gets worse — a burst drops packets 120, 121, and 122 together. In-band FEC in each surviving packet carries only the single previous frame, so packet 123 lets the decoder recover frame 122, but frames 120 and 121 have no surviving redundancy: the packets that carried their recovery copies (121 and 122) were themselves lost in the burst. The FEC decoder recovers what it can (122) and hands 120 and 121 to the PLC fallback, which synthesizes concealment audio from the frames on either side. The result is a brief, slightly muffled patch rather than a jarring dropout — FEC shrank a three-frame gap to a two-frame concealment, and the concealer handled the residue. The receiver's report back to the sender records the elevated loss.
That report closes the adaptive loop. The sender sees loss jump from two percent to eight and burst-shaped, and responds by increasing protection: it switches to a stronger FEC mode that carries redundancy spanning more than one prior frame, or bumps the RED offset so the backup copy is spaced further from the primary, so a single burst is less likely to swallow both a packet and its recovery. The transmitted bitrate rises to pay for this. Over the next second, as the recovered-fraction climbs and concealed frames fall, the call stabilizes. When the Wi-Fi clears and reported loss drops back toward zero, the sender walks the redundancy back down, reclaiming the bandwidth so the codec can spend those bits on higher primary audio quality instead.
Finally, watch the delay budget stay honored throughout. Every recovery the FEC decoder performs happens within the jitter buffer's existing hold window — the redundant copy of a frame rides in a packet that arrives at most a frame or two later, which the buffer was already waiting for. Nothing in the recovery path adds a round trip. If the buffer had to grow to catch later recovery copies, the controller would weigh that added delay against the quality gain and cap it, because a call that recovers every packet but arrives three hundred milliseconds late has traded one impairment for a worse one. The steady state is a buffer just deep enough to let FEC do its work, redundancy just high enough to keep concealment rare, and total mouth-to-ear delay under the threshold where conversation still feels natural — the loop of detect loss, add redundancy, recover, and relax running continuously.