On October 28, 2026, Meta's engineering blog described the next generation of Andromeda, the system that picks a few thousand candidate ads from a much larger pool before heavier ranking models run. Meta says it now scores every eligible ad and "surfaces the top candidates at over one billion ads per second, a new throughput benchmark for ads retrieval-stage models", which it calls a 100x lift over the 2024 Andromeda system. That system, Meta says, evaluated "tens of millions of ads per second per GPU card". Both run on the NVIDIA GH200 Grace Hopper Superchip.
The bottleneck moved from arithmetic to memory
The post's central argument is that at billions of ads per second, compute is no longer the limit. "The memory bandwidth of passing ad tensors through memory hierarchy becomes the new bottleneck." So Meta fused embedding gathering, the interaction layer and scoring into a single GPU kernel. Intermediate results stay on chip instead of being written to and read back from HBM. Meta reports model inference at over 10 billion ads per second on a single GH200 GPU.
Meta and NVIDIA then optimized the gating network that routes each candidate to the relevant user experts. That kernel went from 2.4 billion to over 10 billion ads per second, a 5.3x gain, and reached about 85% of the GPU's peak HBM bandwidth. INT8 quantization added up to 2x more throughput over the optimized FP16 kernels, with accuracy Meta describes as on par.
A bigger model that costs fewer FLOPs
The model itself changed. A single user tower became an ensemble of specialized user towers, and the interaction layer became an attention-based mixture-of-experts gate that sends each ad candidate to its top-K relevant experts. Meta says retrieval quality "approaches ranking-level" at "approximately 300x fewer inference-time FLOPs" than the 2024 model. Routing lets the model grow without every candidate paying for every expert.
Move the pipeline, not the data
The other change is placement. Index matching, scoring and top-K selection now run on the same GPU server. Clients send a small request, and the server returns only the final top-K results. Meta says this "reduces network traffic by ~100x" and returns the system to being limited by GPU compute. Index matching itself reached over 100 billion ads per second, which leaves headroom over the scoring network. For business impact, Meta says that since 2025 Andromeda "has delivered a 2.4% increase in conversions on Facebook".
What remains uncertain. The 10 billion per second figures are kernel measurements in isolation. For the full pipeline, Meta reports only "multiple times" current production throughput, not an absolute number. Retrieval quality is described, not given as recall or ranking metrics, and the 300x FLOPs comparison is not explained. The conversion figure covers Facebook only and gives no method. Work on Grace Blackwell is described as in progress.
Analysis: The lesson travels beyond ads. When a retrieval stage gets fast enough, the cost moves to moving tensors and moving requests. A small recommender for Belgavi's Book Store, picking a few books to show Geetha from a large catalogue, would hit the same wall at much smaller scale: shipping candidate lists between services can cost more than scoring them. Measure bandwidth use before buying more compute, keep candidate generation and scoring close together, and treat any kernel-only benchmark, including Meta's, as an upper bound for the whole pipeline.
Sources and evidence
Meta's October 28 post says next-generation Andromeda surfaces candidates at over one billion ads per second on NVIDIA GH200, 100x its 2024 system, by fusing embedding gathering, interaction and scoring into one kernel and running index matching, scoring and top-K on one server (network traffic cut ~100x). The 10 billion ads/sec figures are isolated kernel benchmarks; end-to-end gains are given only as a multiple of production.