This repository provides the official code for the "LLM Engineer's Handbook," offering a practical guide for building, deploying, and monitoring LLM and RAG applications on AWS using LLMOps best practices. It targets engineers and researchers looking to implement end-to-end LLM systems, from data collection to production-ready deployment and monitoring.

How It Works

The project follows a Domain-Driven Design (DDD) approach, structuring the core Python package (llm_engineering) into domain, application, model, and infrastructure layers. It leverages ZenML for orchestrating ML pipelines, enabling modularity and reproducibility across the ML lifecycle. Key components include data crawlers (Selenium-based), data processing, LLM training (SFT/DPO), RAG implementation, and deployment to AWS SageMaker, with integrations for experiment tracking (Comet ML) and prompt monitoring (Opik).

Quick Start & Requirements

• Install: Clone the repo, set up Python 3.11 with pyenv, install dependencies using poetry install --without aws, and activate the environment with poetry shell.
• Prerequisites: Python 3.11, Poetry >= 1.8.3, Docker >= 27.1.1, AWS CLI >= 2.15.42, Git >= 2.44.0. Requires API keys for OpenAI and Hugging Face, and optionally for Comet ML.
• Setup: Local infrastructure (MongoDB, Qdrant) can be started with poetry poe local-infrastructure-up. Cloud deployment requires AWS configuration.
• Docs: LLM Engineer's Handbook

Highlighted Details

• End-to-end LLM system development: data collection, training, RAG, AWS deployment, monitoring, and testing.
• Domain-Driven Design principles applied to LLM engineering.
• ZenML for ML pipeline orchestration and reproducibility.
• Production-ready deployment to AWS SageMaker.
• Integrations with Comet ML for experiment tracking and Opik for prompt monitoring.

Maintenance & Community

The repository is maintained by Packt Publishing and the authors Paul Iusztin and Maxime Labonne. The README encourages users to check the GitHub Issues section for troubleshooting and to ask for help.

Licensing & Compatibility

Released under the MIT license, allowing for free use, modification, and distribution, provided attribution is given.

Limitations & Caveats

The code is actively maintained and may differ from the book's content. Running the project incurs costs, primarily for AWS SageMaker and OpenAI API usage (estimated ~$25 for a single run). LinkedIn and Medium data crawlers require a Chromium-based browser (like Chrome) or Docker.