This repository serves as a curated collection of resources for GPU programming, primarily focusing on CUDA. It targets developers, researchers, and students interested in high-performance computing, AI acceleration, and kernel development, offering a centralized hub for learning materials and practical implementations.

How It Works

The project aggregates links to books, papers, blog posts, videos, and code repositories related to CUDA, Triton, PyTorch performance, and GPU architecture. It aims to provide a structured learning path from introductory concepts to advanced optimization techniques and research frontiers in GPU computing.

Quick Start & Requirements

• Contribution: Submit links via pull request or Discord.
• Learning: Primarily relies on external resources (links provided).
• Prerequisites: Access to Discord and YouTube for live sessions and recordings.

Highlighted Details

• Comprehensive coverage from "1st Contact with CUDA" to advanced topics like CUDA Graphs and Tensor Core programming.
• Links to key libraries and tools: CUTLASS, Triton, PyTorch C++ API, Numba, TVM, JAX, CuPy, Codon.
• Includes resources on foundational AI systems, data-centric AI, and specific optimizations like FlashAttention-2.
• Features "CUDA Grandmasters" like Tri Dao and Tim Dettmers, with links to their influential work.

Maintenance & Community

• Active community via Discord server (link provided).
• Regular lectures and reading group sessions with recordings on YouTube.
• Contributions are encouraged via pull requests to the repository.

Licensing & Compatibility

• Resource links point to various external sources with their own licenses.
• Code examples and libraries mentioned have their respective licenses (e.g., MIT, Apache 2.0).

Limitations & Caveats

This is a curated list of links, not a runnable software project. The quality and availability of external resources are subject to their original sources. Some older links or specific implementations might be outdated.