Core concept
Default stream (0) synchronous. Non-default streams async. cudaStreamCreate + cudaMemcpyAsync.
Advertisement
How it works
Overlap compute + memory transfer. Requires page-locked host memory.
Advertisement
Trade-offs + gotchas
Data pipeline (load next batch while compute current). Multi-kernel overlap.