This project demonstrates running Large Language Models (LLMs) entirely within a PDF file, targeting developers and researchers interested in novel execution environments for AI. It enables LLM inference directly in a PDF viewer, offering a unique and portable way to interact with AI models.

How It Works

The core innovation lies in compiling llama.cpp to asm.js using Emscripten. This compiled JavaScript code is then embedded within a PDF file, leveraging an older PDF.js injection technique. The LLM model itself, quantized in GGUF format, is base64 encoded and embedded directly into the PDF, allowing for self-contained inference. This approach bypasses traditional execution environments, making the LLM accessible solely through a PDF reader.

Quick Start & Requirements

• Install and run via Python script: cd scripts && python3 generatePDF.py --model "path/for/model.gguf" --output "path/to/output.pdf"
• Prerequisites: Python 3, llama.cpp compatible GGUF quantized models (Q8 recommended for speed).
• Model size: 135M parameter models yield ~5s per token. Larger models are expected to be significantly slower.
• Further details: YouTube video linked in README.

Highlighted Details

• Proof-of-concept for running LLMs within PDF files.
• Utilizes Emscripten to compile llama.cpp to asm.js.
• Employs PDF.js injection for execution.
• Supports GGUF quantized models, with Q8 recommended for performance.

Maintenance & Community

• Project appears to be a personal project by EvanZhouDev.
• No explicit community channels or roadmap are mentioned in the README.

Licensing & Compatibility

• The README does not specify a license.

Limitations & Caveats

This is a proof-of-concept with significant performance limitations; larger models are impractical due to slow inference speeds. Compatibility may depend on specific PDF viewer versions and their JavaScript execution capabilities.