This repository offers comprehensive lecture notes and resources for understanding deep learning, emphasizing observed phenomena and their theoretical underpinnings. It targets students, researchers, and engineers seeking a deep dive into DL's core concepts, from fundamental mechanisms like backpropagation and optimization to advanced topics such as the frequency principle, condensation phenomenon, and neural network analysis of LLMs and scientific computing. The project provides a structured learning path with video lectures and practical Jupyter notebooks, aiming to bridge engineering practice and theoretical understanding.

How It Works

The project systematically explains deep learning by presenting phenomena and dissecting them mathematically. It covers neural network architectures (CNNs, Transformers), optimization, generalization, and the curse of dimensionality. Advanced chapters explore the frequency principle (NNs learn low-frequency first), condensation (neuron convergence), loss landscape embedding, and initialization impacts. It also analyzes LLMs via "Anchor Functions" and applies NNs to solve differential equations, noting their unique frequency biases compared to traditional methods.

Quick Start & Requirements

• Installation: Primarily lecture notes; experimental code provided via Jupyter notebooks. Setup involves a standard Python environment.
• Prerequisites: Python, Jupyter Notebooks. Libraries like NumPy, PyTorch/TensorFlow are likely required for notebooks. GPU recommended for experiments.
• Links: Course videos available on Bilibili (search for 许志钦); experimental notebooks included within the repository.

Highlighted Details

• Phenomenon-Driven Approach: Prioritizes empirical observations for understanding DL.
• Frequency Principle: Detailed theoretical and experimental analysis of low-frequency bias in NNs.
• Condensation Phenomenon: Explains implicit complexity reduction in wide networks through neuron convergence.
• LLM Analysis Framework: Introduces "Anchor Functions" for studying LLM reasoning and internal mechanisms.
• AI for Science: Dedicated chapters on solving differential equations with NNs, highlighting frequency biases.
• Comprehensive Theory: Covers topics like Barron space, embedding principles of loss landscapes, optimistic estimates, and solution flatness.

Maintenance & Community

• Contact: xuzhiqin@sjtu.edu.cn
• Videos: Available on Bilibili (search for 许志钦).
• Activity: Recent updates (June 2025, Nov 2024) indicate ongoing maintenance.

Licensing & Compatibility

• License: Not specified in the provided README, a critical omission for adoption decisions.
• Compatibility: Assumed to be standard Python environments; no specific compatibility notes are provided.

Limitations & Caveats

• Language Barrier: README is primarily in Chinese, potentially limiting accessibility for non-Chinese speakers.
• Licensing Uncertainty: The absence of a clear license prevents definitive statements on commercial use or redistribution.
• Setup Effort: Running experiments requires setting up a Python environment and managing dependencies for Jupyter notebooks.
• Focus: Primarily lecture notes and theoretical exploration, not a production-ready library.