P2P KV Cache Transfer

Why it matters

P2P KV transfer enables disaggregated + prefix sharing. Understanding shapes infra.

The architecture

Source GPU: KV cache resident.

Transport: NVLink / InfiniBand.

Dest GPU: receives.

How it works end to end

NCCL or UCX-based.

GPU-direct RDMA over IB.

NVLink intra-node.

Overlap with decode.