This repository provides code for LLM-infused risk-sensitive reinforcement learning for trading agents, targeting researchers and practitioners in quantitative finance. It offers novel approaches to enhance trading agent performance by integrating Large Language Models (LLMs) for risk-aware decision-making, aiming for improved financial strategies.

How It Works

The project integrates LLM-generated sentiment and risk signals into traditional reinforcement learning algorithms like PPO and CPPO. This is achieved by preprocessing financial news data with LLMs (DeepSeek) to extract sentiment and risk metrics, which are then incorporated into the trading environment. This LLM-infused approach allows agents to make more nuanced, risk-sensitive trading decisions compared to standard RL methods.

Quick Start & Requirements

• Installation: Run installation_script.sh on an Ubuntu server.
• Prerequisites: 128 GB RAM CPU instance recommended. Requires specific datasets available on Hugging Face.
• Links:
  • Paper: https://arxiv.org/abs/2502.07393
  • Data: https://huggingface.co/datasets/benstaf/nasdaq_2013_2023/tree/main
  • Backtesting Notebook: FinRL_DeepSeek_backtest.ipynb

Highlighted Details

• Preliminary results suggest PPO performs better in bull markets, while CPPO-DeepSeek excels in bear markets.
• LLM signals are generated using sentiment_deepseek_deepinfra.py and risk_deepseek_deepinfra.py.
• Training commands are provided for PPO, CPPO, PPO-DeepSeek, and CPPO-DeepSeek.
• Evaluation metrics include Information Ratio, CVaR, and Rachev Ratio.

Maintenance & Community

The project has been integrated into the original FinRL project by AI4Finance and is the basis for task 1 in the FinRL contest 2025.

Licensing & Compatibility

The repository's license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The README recommends specific hardware (128 GB RAM CPU instance) for installation, suggesting potentially high resource requirements. The LLM integration involves multiple data preprocessing steps, which may require significant computational resources and time. The specific LLM models and their integration details might require further investigation for optimal performance.