Why it matters
CUDA remains the dominant GPU programming interface. Every ML framework compiles down to CUDA kernels for NVIDIA GPUs. Understanding it matters for performance debugging and custom kernel development.
The architecture
Kernel: a __global__ function launched from the host (CPU) that runs on the device (GPU). Called with launch configuration <<<grid, block>>> specifying how many threads to launch.
Thread hierarchy: threads group into blocks (up to 1024 threads); blocks group into a grid (up to millions of blocks). Threads within a block can share memory and synchronize; threads across blocks cannot.
How it works end to end
Kernel invocation: 'add<<<numBlocks, threadsPerBlock>>>(a, b, c)'. Runtime launches numBlocks × threadsPerBlock total threads.
Thread identification: threadIdx.x/y/z within block, blockIdx.x/y/z within grid, blockDim.x/y/z gives block size. Use these to compute global thread index.
Memory: cudaMalloc allocates device memory; cudaMemcpy transfers between host and device. Modern APIs (unified memory) hide the transfer but perf may suffer.