Why architecture matters here
Echo is uniquely corrosive because it punishes the other party. The person whose device leaks echo hears nothing wrong; the far end hears themselves 100-300ms late, which is precisely the delay range that makes fluent human speech neurologically difficult. Ten seconds of it and people start stuttering; a minute and they end the call. For voice agents the stakes are different but just as hard: barge-in — letting a user interrupt the assistant mid-sentence — only works if the assistant's own TTS output is removed from the microphone signal before the speech recognizer sees it. Without AEC, the agent transcribes its own voice, feeds it back into the dialog model, and the conversation collapses into self-talk.
The naive fix — half-duplex gating, where the mic is muted while the speaker plays — is what speakerphones did in the 1990s and what bad conferencing software still effectively does with over-aggressive suppression. It makes interruption impossible and clips the first syllable of every response. Full duplex requires actually modeling the echo: the speaker-to-mic transfer function is a linear system (mostly), so a filter that learns it can predict the echo sample by sample and subtract it, leaving simultaneous near-end speech untouched.
The architecture matters because that model lives under brutal constraints. It must run on 10ms frames with a few milliseconds of compute budget, on hardware from flagship phones to $30 smart speakers. It must track a physical echo path that changes whenever the user moves the device, turns up the volume, or walks into another room. And it must never confuse near-end speech with echo — the failure that makes a canceller 'eat' the user's voice. Every design decision below exists to hold those three requirements simultaneously.
The architecture: every piece explained
Top row of the diagram: the echo's journey. Far-end audio arrives from the network, exits the jitter buffer, is decoded to PCM, and enters the OS render path. The render buffer is where the reference tap must live: the canceller needs the samples as actually played, after any OS mixing, per-app volume, and system sounds are applied — which is why serious implementations use loopback capture APIs (WASAPI loopback, CoreAudio taps, AAudio) rather than the pre-mix application signal. The speaker converts those samples to air pressure, the echo path — direct coupling through the device chassis plus room reflections — shapes them, and the microphone captures the sum of near-end speech, echo, and ambient noise.
Middle row: the linear canceller. The delay estimator cross-correlates the reference against the microphone signal to find the bulk delay — typically 40-200ms of buffering plus acoustic flight time — and re-aligns the two streams; modern cancellers track this continuously because delays drift. The adaptive filter is almost always a partitioned-block frequency-domain NLMS variant: the echo path's impulse response (100-250ms of taps) is split into blocks, each adapted in the frequency domain, giving near-RLS convergence at FFT cost. The filter's output ŷ(n) is the predicted echo; subtraction yields the error signal e(n), which is both the output and the adaptation signal.
Bottom rows: protection and cleanup. The double-talk detector compares near-end energy and coherence against the predicted echo; when the near end is speaking, adaptation is slowed or frozen so the filter does not learn to cancel the user's voice. The residual echo suppressor handles what linearity misses — speaker distortion at high volume, chassis vibration, quantization — by estimating residual echo power per frequency band and applying spectral gains, increasingly with a small neural network doing the estimation. Downstream, noise suppression and AGC run on the cleaned signal, and telemetry publishes ERLE (echo return loss enhancement) and delay stability so regressions are visible in dashboards rather than app-store reviews.
End-to-end flow
Follow one 10ms frame at 48kHz — 480 samples — through the chain. The far end says a word; its PCM lands in the render buffer at time T. The reference tap copies those 480 samples into a ring buffer keyed by presentation timestamp. The samples play, cross the room, and arrive back in the capture stream roughly 60ms later, mixed with whatever the near-end user is doing. The capture callback fires with its own 480 samples, and the canceller must now answer: which reference frame does this capture frame correspond to?
The delay estimator answers by correlating recent capture against a window of the reference ring buffer, usually on downsampled or spectrally-whitened signals for robustness. Say it reports 64ms: the canceller reads the reference from 64ms ago, runs it through the adaptive filter's current coefficients, and produces a predicted echo frame. Subtraction gives e(n). Now the double-talk decision: if e(n)'s energy is small relative to the prediction and coherence between reference and capture is high, this frame is echo-dominated — adapt at full step size. If a near-end voice detector fires or coherence drops, freeze adaptation and just subtract with the coefficients you have.
From a cold start — a call's first seconds — the filter converges over roughly 0.5-1.5 seconds of far-end speech, which is why the first sentence of a call sometimes leaks a little echo. Converged, the linear stage typically removes 20-30dB of echo. The residual suppressor then estimates remaining echo power per band (using the filter's own prediction as a cue), applies gains that may attenuate another 15-25dB during far-end-only periods, relaxes during double-talk to preserve duplexity, and inserts low-level comfort noise so suppression does not sound like the line going dead. The cleaned frame flows to noise suppression, AGC, and the encoder — total added latency: a few milliseconds.