This project provides a working example of Retrieval Augmented Generation (RAG) for financial applications using Llama 2 70b and Llama Index. It is targeted at developers and researchers looking to implement RAG systems, offering a practical demonstration for building finance-focused chatbots.

How It Works

The system leverages Llama Index for efficient data indexing and retrieval, combined with the Llama 2 70b chat model for generating responses. This approach allows the LLM to access and synthesize information from a financial knowledge base, enabling more accurate and contextually relevant answers.

Quick Start & Requirements

• Install via git clone https://github.com/nicknochnack/Llama2RAG and cd Llama2RAG.
• Run the Jupyter notebook by executing jupyter lab.
• Alternatively, run the Streamlit app with streamlit run app.py.
• Requires a Hugging Face authentication token.
• Highly recommended: GPU acceleration (e.g., A100-80GB GPU).
• References: Llama 2 70b Chat Model Card, Llama Index Docs

Highlighted Details

• Demonstrates RAG for financial use cases.
• Utilizes Llama 2 70b and Llama Index.
• Includes a Streamlit application for interactive use.

Maintenance & Community

• Author: Nick Renotte.
• Version: 1.x.

Licensing & Compatibility

• MIT License. Permissive for commercial use and closed-source linking, with a general ethical use clause.

Limitations & Caveats

The project strongly recommends GPU acceleration, implying potential performance issues or unfeasibility on CPU-only environments.