All 42 articles, sorted alphabetically
Adaptive Jitter Buffer Design, in depth: building it as a testable component, trace-driven tuning, and the traffic patterns that break it
An engineering guide to implementing an adaptive jitter buffer rather than describing one: the component boundary between the network and the audio cl…
Read article →AAC, in depth: the MDCT filterbank, psychoacoustics, quantisation loops, SBR and the bitstream headers you will actually debug
How Advanced Audio Coding works from first principles: the MDCT and window switching, the psychoacoustic model and scalefactor bands, the rate and dis…
Read article →Acoustic echo cancellation architecture, in depth: reference, delay, adaptive filter, double talk and residual suppression
How an acoustic echo canceller is built: signal model, loopback reference and clock drift, GCC-PHAT delay estimation, partitioned-block FD-NLMS, doubl…
Read article →Automatic Gain Control (AGC) for Voice Audio
How automatic gain control works for voice: level estimation, gain loop, fast attack and slow release, noise gate, look-ahead limiting, and avoiding p…
Read article →Automatic Speech Recognition Architecture, in depth: front-end, encoder, CTC vs transducer vs attention decoders, decoding and honest WER
How a modern automatic speech recognition system is built: log-mel features and subsampling, Conformer and Transformer encoders, the three decoder fam…
Read article →Microphone beamforming architecture, in depth: array geometry, delay-and-sum, MVDR and superdirective designs, steering, and where it sits next to echo cancellation
How microphone-array beamformers work and how to build one: steering vectors, spatial aliasing and aperture, delay-and-sum, MVDR and superdirective we…
Read article →Audio Buffer Management for Low Latency, in depth: periods, callback deadlines, lock-free rings and xrun-driven tuning
How to get audio latency down without glitches: building the latency budget from frames and sample rate, period and buffer sizing, the audio callback …
Read article →Audio Clock Drift Compensation: Adaptive Resampling, PI Loop
Why sender and receiver audio clocks drift by parts per million, how to measure it from jitter buffer depth, and how a PI loop drives adaptive resampl…
Read article →Audio Codecs, in depth: how perceptual transform codecs work from the MDCT to the bitstream, with a runnable toy codec, speech versus music designs, latency budgets, quality measurement and failure modes
A first-principles guide to how audio codecs compress sound: codec families, the MDCT filterbank and time-domain aliasing cancellation, block switchin…
Read article →Audio dereverberation architecture
Deep-dive on speech dereverberation: why reverberation is convolution not additive noise, STFT analysis, late-reflection estimation with weighted pred…
Read article →Speaker diarization architecture, in depth: the system around the model, from long-audio jobs and speaker linking to enrolled identities, evaluation and voiceprint privacy
How to build a production speaker diarization system: a job pipeline, splitting long audio into blocks and linking speakers between them, open-set ide…
Read article →Speaker diarization v2 architecture, in depth: overlap-aware neural segmentation, speaker embeddings, constrained clustering and reconstruction
How modern hybrid speaker diarization works: why window-and-cluster pipelines fail on overlap, local powerset segmentation with permutation-invariant …
Read article →DTX and Comfort Noise Explained: VAD, SID Frames, Silence
Discontinuous transmission (DTX) and comfort noise generation in voice codecs: VAD, silence insertion descriptor (SID) frames, receiver noise synthesi…
Read article →Audio forward error correction
Deep-dive on FEC and redundancy for real-time audio: why retransmission fails against the conversational deadline, in-band Opus FEC and RED-style redu…
Read article →Acoustic howling suppression architecture
Deep-dive on howling/feedback suppression: loop-gain stability and maximum/added stable gain, feedback detection and notch filtering, frequency shifti…
Read article →Adaptive jitter buffer architecture, in depth: the receive pipeline, decoder hand-off, mixing, lip sync and metrics
Adaptive jitter buffer architecture in a real receiver: push and pull halves, the device callback contract, Opus PLC, FEC and DTX, per-stream mixing, …
Read article →Loudness normalization architecture
Deep-dive on loudness normalization: why peak normalization fails perceptually, the ITU-R BS.1770 chain (K-weighting, 400ms block mean-square, absolut…
Read article →Multi-party audio mixing architecture
Deep-dive on server-side multi-party audio mixing (the MCU): per-stream jitter buffering and decoding, resampling to a common clock to correct device …
Read article →Music Generation Architecture, in depth: codec tokens and latents, autoregressive and diffusion generators, conditioning, serving and safeguards
How text-to-music systems are built: why they generate compressed representations instead of samples, codec tokens versus continuous latents, autoregr…
Read article →Neural Audio Codecs: Audio Tokens to Waveform
How neural audio codecs work: encoder, residual vector quantization (RVQ) and decoder, audio tokens for TTS and audio LLMs, decoding tokens to wavefor…
Read article →Real-time noise suppression architecture, in depth: capture-chain order, band-gain masking, the per-frame loop and deadlines
How real-time noise suppression works: placement after echo cancellation, bounded band-gain masking as in RNNoise, a causal per-frame STFT loop in cod…
Read article →Opus codec architecture
Deep-dive on the Opus audio codec: SILK speech and CELT music engines, hybrid mode and content-adaptive switching, flexible frame sizes, FEC and PLC p…
Read article →Packet loss concealment
Deep-dive on packet loss concealment (PLC) for real-time voice: manufacturing replacement frames for lost packets by extrapolating from recent history…
Read article →PCM Audio, in depth: samples, bit depth, layouts, WAV files and dither
PCM audio from first principles: sampling and Nyquist, bit depth and the 6.02N + 1.76 dB rule, integer and float sample formats, interleaving and endi…
Read article →Real-Time Audio Architecture in Depth
Real-time audio architecture in depth: the audio callback contract, lock-free ring buffers, AEC/NS/AGC/VAD, Opus framing, FEC, RED and PLC, jitter and…
Read article →Sample-rate conversion architecture
Deep-dive on audio sample-rate conversion (resampling): how to move audio between rates like 48 kHz, 44.1 kHz, and 16 kHz without aliasing or imaging.…
Read article →Audio Source Separation Architecture, in depth: masking, the phase problem, Conv-TasNet, Demucs, PIT training, evaluation and streaming inference
How audio source separation works and how to build and run it: the mixture model, time-frequency masks and the phase problem, learned-basis models lik…
Read article →Spatial audio -- placing sound in 3D space
Deep-dive on spatial audio: the brain's localization cues (ITD, ILD, spectral), the HRTF, binaural rendering, object-based audio and ambisonics, …
Read article →Streaming ASR architecture
Deep-dive on streaming speech recognition: VAD, streaming encoder, CTC/transducer, partials, endpointing, rescore, and observability.
Read article →Audio Streaming Protocols Compared, in depth: HTTP, Icecast, HLS, DASH, WebRTC and ingest
How audio streaming protocols differ: progressive HTTP, Icecast, HLS and DASH, low-latency variants, WebRTC with SFUs, RTMP, SRT and WHIP ingest, late…
Read article →Audio Super-Resolution: Neural Bandwidth Extension Explained
How audio super-resolution (bandwidth extension) restores missing high frequencies: narrowband to wideband, GAN and diffusion models, and hallucinatio…
Read article →Text-to-Speech Architecture, in depth: text front end, acoustic models, vocoders, codec language models and streaming
How modern TTS systems are built: the text normalization and G2P front end, acoustic model families from Tacotron 2 and FastSpeech 2 to VITS, codec la…
Read article →Voice Activity Detection (VAD): Silero, WebRTC, Endpointing
How voice activity detection (VAD) works: framing, energy, WebRTC GMM and Silero neural scoring, hysteresis, hangover and pre-roll, ASR gating and bar…
Read article →Wake Word Detection Architecture, in depth: always-on cascades, streaming keyword models, posterior smoothing and the false-accept budget
How wake word (keyword spotting) systems are built: the false-accept and false-reject metrics, a low-power multi-stage cascade, a streaming log-mel fr…
Read article →Echo Cancellation for Speakerphone, in depth: why hands-free is the hardest AEC case, and how to build, tune and test it
Acoustic echo cancellation for speakerphones and hands-free devices: loud coupling and negative echo-to-speech ratio, loudspeaker nonlinearity, the re…
Read article →Jitter Buffer Design, in depth: delay estimation, the playout decision, late versus lost packets, and time stretching
How to design an adaptive audio jitter buffer for real-time voice: RTP timing, why the RFC 3550 jitter value is not a sizing rule, a quantile-based de…
Read article →MP3 vs AAC vs Opus for Streaming, in depth: compatibility, quality per bit, containers, gapless playback and a decision guide
How to choose between MP3, AAC and Opus for on-demand and live audio streaming: who can decode each, how quality holds up as bitrate falls, containers…
Read article →The Opus Codec for Real-Time Voice, in depth: encoder settings, RTP and SDP, in-band FEC, DTX and the latency budget
How to run Opus in a live voice path: libopus encoder and decoder API, frame size and latency, bitrate and bandwidth for speech, RFC 7587 RTP and SDP …
Read article →Opus Codec Parameters That Matter, in depth: bitrate modes, complexity, bandwidth, frame size and stereo across calls, games, music and files
A parameter-by-parameter guide to the Opus encoder: what is set at creation, VBR versus constrained VBR versus CBR, complexity, bandwidth caps, frame …
Read article →Real-Time Audio Pipelines in Rust, in depth: deadlines, thread roles, cpal streams, lock-free ring buffers and glitch-free control
How to build real-time audio pipelines in Rust: buffer-size deadline arithmetic, the three thread roles, what is forbidden in the audio callback, sett…
Read article →Sample Rate Conversion Techniques, in depth: from linear interpolation to windowed sinc, polyphase, FFT and arbitrary-ratio resamplers, and how to measure what each costs you
A practical guide to choosing a sample rate converter: why resampling aliases, the ladder of techniques from drop-sample to windowed sinc, rational po…
Read article →WebRTC Audio Pipeline End to End, in depth: capture, echo cancellation, Opus, RTP, the jitter buffer and playout, with a latency budget and the stats that diagnose it
Follow one voice frame through a WebRTC call: device capture, the audio processing module with AEC, noise suppression and AGC, Opus encoding with FEC …
Read article →