This repository contains materials for a graduate seminar on Large Language Models (LLMs) at UNC Chapel Hill. It's designed for students with a machine learning and NLP background, offering a structured approach to understanding LLM history, advancements, and applications through a role-playing seminar format. The primary benefit is developing deep comprehension of LLM research papers and enhancing critical analysis and presentation skills.

How It Works

The course utilizes a role-playing seminar format where students present and discuss foundational and recent LLM research papers. Each session focuses on two complementary papers, with students assigned roles like Reviewer, Archaeologist, Hacker, or Diagrammer. This structure encourages active engagement, diverse perspectives, and hands-on understanding of LLM concepts through implementation or visualization.

Quick Start & Requirements

• Prerequisites: Experience with machine learning (preferably deep learning) and modern natural language processing. Ability to understand recent ML/NLP conference papers.
• Course Structure: Role-playing seminar with paper presentations and discussions. Grading based on presentations and participation.
• Resources: The README provides a detailed schedule of papers, presentation roles, non-presenter assignments, and grading criteria.

Highlighted Details

• Covers seminal papers like "Attention Is All You Need," BERT, GPT-NeoX, and BLOOM.
• Includes roles for implementing parts of papers ("Hacker") and contextualizing research ("Archaeologist").
• Features discussions on LLM scaling, ethics ("Stochastic Parrots"), and evaluation.
• Provides a comprehensive schedule of topics and readings for Fall 2022.

Maintenance & Community

• Maintained by Colin Raffel, an instructor at UNC Chapel Hill.
• The repository serves as a static archive of course materials; no active community or development is indicated.

Licensing & Compatibility

• The repository itself is not explicitly licensed. The content (course materials) is likely subject to UNC Chapel Hill's academic policies.
• The papers discussed are under their respective publisher/author licenses.

Limitations & Caveats

This repository is a static archive of a past seminar and does not contain code for running LLMs or a platform for ongoing discussion. It is purely educational material.