Why it matters
Bad launch configuration can leave 80 percent of GPU idle. Understanding occupancy and launch parameters is core to GPU programming.
The architecture
Launch config: kernel<<<grid, block, sharedMem, stream>>>(args). Grid is number of blocks (up to 2^31). Block is threads per block (up to 1024).
Block size affects occupancy. Common sizes are 128, 256, 512. Compute Occupancy Calculator tells you if you'll fill the SM.
How it works end to end
Occupancy: SMs can host multiple blocks concurrently, up to hardware limits. High occupancy hides memory latency because SM has other warps to run when one stalls.
CUDA streams: independent execution queues. Kernels in different streams can run concurrently if resources allow.
Cooperative launches: threads across the entire grid can synchronize via cooperative groups. Enables kernel-level algorithms like parallel scan.