grokking-pytorch  by Kaixhin

PyTorch guide with notes on usage, best practices, and debugging

created 7 years ago
1,197 stars

Top 33.4% on sourcepulse

GitHubView on GitHub
Project Summary

This repository provides a detailed, tutorial-like guide to understanding and using PyTorch, focusing on its imperative programming style and flexibility for dynamic neural networks. It's aimed at users who have completed basic PyTorch tutorials and want a deeper dive into practical implementation details, including GPU acceleration, data loading, and model checkpointing.

How It Works

The guide walks through building and training a Convolutional Neural Network (CNN) for the MNIST dataset. It emphasizes PyTorch's dynamic computation graph, which allows for imperative programming, making code easier to read and debug compared to static graph frameworks. Key aspects covered include device-agnostic code for GPU/CPU utilization, efficient data loading with DataLoader and num_workers, and the use of nn.Module for defining network layers and optim for parameter optimization.

Quick Start & Requirements

  • Install: pip install torch torchvision
  • Prerequisites: Python 3.x, CUDA (optional, for GPU acceleration).
  • Run: Execute the provided Python script.
  • Resources: Requires downloading the MNIST dataset. GPU acceleration is recommended for faster training.
  • Docs: Official PyTorch Tutorials

Highlighted Details

  • Demonstrates device placement (.to(device)) for seamless CPU/GPU operation.
  • Explains DataLoader optimizations like num_workers and pin_memory.
  • Details model.train() vs. model.eval() and torch.inference_mode().
  • Covers gradient accumulation, manual zeroing (optimizer.zero_grad()), and checkpointing (state_dict).

Maintenance & Community

This repository appears to be a personal learning resource rather than an actively maintained project. No specific community links or contributor information are provided in the README.

Licensing & Compatibility

The repository itself does not specify a license. The code examples use PyTorch, which is typically licensed under a BSD-style license, allowing for commercial use and integration into closed-source projects.

Limitations & Caveats

The README explicitly states that this guide does not cover all parts of PyTorch or every best practice. The code is presented in chunks for readability, not as modular, production-ready code. It's a learning resource, not a library.

Health Check
Last commit

3 years ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
1 stars in the last 90 days

Explore Similar Projects

Starred by Stas Bekman Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake).

fms-fsdp by foundation-model-stack

0.4%
258
Efficiently train foundation models with PyTorch
created 1 year ago
updated 1 week ago
Starred by Peter Norvig Peter Norvig(Author of Artificial Intelligence: A Modern Approach; Research Director at Google), Didier Lopes Didier Lopes(Founder of OpenBB), and
15 more.

llm.c by karpathy

0.2%
27k
LLM training in pure C/CUDA, no PyTorch needed
created 1 year ago
updated 1 month ago
Feedback? Help us improve.