Learn
CUDA Parallel Programming
Write CUDA kernels chapter by chapter: from turning a loop into your first kernel to convolution, reduction, prefix sums, sorting and faster matrix multiplication.
Start stage 1 →Part 1 GPU Programming Foundations
From Loops to Kernels
- 1Chapter Notes and QuizData parallelism, a thread's global index and block, the i < n bounds check, launch configuration, and moving data to and from the GPU.Quiz
- 2Celsius to FahrenheitCompute each thread's global index and guard extra threads with i < n.EasyCoding
- 3Two Elements per ThreadHalf as many threads as elements: each thread covers two, checking the second against n.EasyCoding
Threads over 2D and 3D Data
- 4Chapter Notes and QuizComing soon
- 5Matrix TransposeFind each element's position in the input and in the output in row-major order, and write a 2D grid that covers the whole matrix.EasyCoding
- 6Video Fade-OutComing soon
- 7Weighted ScoresComing soon
- 8One Row per ThreadComing soon
Coming soon
- How a GPU Runs Your Threads9Control Divergence
- Shared Memory and Tiling10Tiled Matrix Multiplication
- Tuning Kernel Performance11Memory Coalescing
Part 2 Classic Parallel Algorithms
Coming soon
- Convolution Filters122D Convolution
- Stencils on Grids133D Stencil
- Histograms and Atomics14Histogram
- Reducing an Array to One Value15Sum Reduction
- Prefix Sums16Inclusive Scan
- Filtering and Stream Compaction17Simple Filter
- Merging Sorted Arrays18Parallel Merge
- Sorting on the GPU19Radix Sort
- Faster Matrix Multiplication20Register Tiling