What Happens Inside a GPU

What happens inside a GPU when a kernel runs? This talk takes you through a busy train station, where cargo handlers represent threads, terminals represent streaming multiprocessors, and cargo moves through the GPU’s memory hierarchy. Using this analogy, we’ll explore how GPUs schedule work, feed Tensor Cores, and overlap data transfers with computation. We’ll also examine why bigger tiles aren’t always faster and why different workloads and GPU generations need different execution strategies. You’ll leave with an intuitive framework for understanding GPU performance, spotting potential bottlenecks, and following deeper discussions of kernel optimization. No prior experience writing GPU kernels is required. https://docs.google.com/presentation/d/e/2PACX-1vRmOW2-A6J4IGgp88QEffOOnFb_XO4HNFNCpMW-06xz2KjYosz_MrvQ36sZfj-E3g/pub?start=false&loop=false&delayms=3000 Slides for past meetups posted: [Github](https://github.com/YanXuHappygela/LLM-reading-group/tree/main) Recordings posted at: [YanAITalk](https://www.youtube.com/@yanaitalk/videos) Feel free to reach out if you want to present at upcoming meetups! Note: You must have a Zoom account to login (free account is sufficient). Zoom Link will be posted to the event page one day before the meetup.