This project provides an experimental Streamlit chatbot application for interacting with LLaMA 2 models, targeting developers and researchers who want a quick way to deploy and test LLM-based conversational agents. It offers a user-friendly interface for selecting different LLaMA 2 model sizes and configuring generation parameters, simplifying the process of experimenting with LLM capabilities.

How It Works

The application leverages Streamlit for its UI and integrates with Replicate to host and serve various LLaMA 2 models (7B, 13B, 70B). Users can select model endpoints and adjust hyperparameters like temperature and top-p directly from the sidebar. The architecture maintains session-specific chat history, providing a conversational context, though history is cleared upon page refresh.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Requires Replicate API token and Auth0 credentials for authentication.
• Run the app: streamlit run llama2_chatbot.py
• Dockerfile is provided for deployment on platforms like Fly.io.
• Official demo: LLaMA2.ai

Highlighted Details

• Supports multiple LLaMA 2 model sizes (7B, 13B, 70B) hosted on Replicate.
• Configurable model hyperparameters (Temperature, Top P, Max Sequence Length).
• Session-based chat history.
• Docker image included for easy deployment.

Maintenance & Community

• Version 0.9.0 (Experimental) released July 2023.
• Contributions are welcome.

Licensing & Compatibility

• Repository License: Apache 2.0
• LLaMA 2 models are subject to Meta Platforms, Inc.'s license agreement.
• Compatible with commercial use under the Apache 2.0 license for the application code.

Limitations & Caveats

This is an experimental application and is provided "as-is" without liability. Chat history is not persistent across sessions or refreshes.