This repository contains modified code and datasets for LLM research, specifically focusing on evaluating Large Language Models' reasoning capabilities on challenging benchmarks like IMO 2023. It targets researchers and engineers interested in quantitative analysis of LLM performance.

How It Works

The project modifies existing codebases to facilitate experimentation with LLMs. It leverages datasets and potentially custom evaluation metrics to assess model performance on complex reasoning tasks, as evidenced by attached screenshots of Llama 3.1 8B and Claude Sonnet solving IMO 2023 problems.

Quick Start & Requirements

• Installation: Not explicitly detailed, likely requires cloning the repository and setting up Python environment.
• Prerequisites: Python, potentially specific libraries for LLM interaction and data handling. GPU acceleration is highly probable for running LLMs.
• Links: No official documentation or quick-start guides are provided.

Highlighted Details

• Demonstrates LLM performance on IMO 2023.
• Includes modified code from bklieger-groq/g1.
• Provides screenshots of model outputs for Llama 3.1 8B and Claude Sonnet.

Maintenance & Community

• No information on contributors, community channels, or roadmap is available.

Licensing & Compatibility

• License information is not provided in the README.

Limitations & Caveats

The repository lacks clear setup instructions, dependency lists, and licensing information, making it difficult to assess reproducibility and compatibility. The primary content appears to be screenshots rather than executable code for direct evaluation.