AI-Writer is a Chinese novel generation tool that leverages the RWKV language model, offering an alternative to GPT-style models with potentially faster inference. It targets users interested in generating fantasy and romance web novels, providing GPU acceleration for NVIDIA and AMD/Intel cards, as well as CPU-only execution.

How It Works

The project utilizes the RWKV (Receptance Weighted Key Value) model, an RNN architecture that aims to combine the parallelizable training of Transformers with the efficient inference of RNNs. This approach allows for a fixed computational cost per token, regardless of sequence length, and a context length of up to 512 tokens. The model predicts the next character based on a probability distribution derived from the preceding context.

Quick Start & Requirements

• Installation: Download pre-trained models from releases and run python run.py or python server.py.
• Prerequisites:
  • Python 3.8 recommended.
  • NVIDIA GPU: CUDA 11.1, cuDNN, torch 1.9.1+cu111.
  • AMD/Intel GPU: pip install torch onnxruntime-directml.
  • CPU: AVX2 support recommended (Intel 4th gen+).
• Resources: Requires ~2GB VRAM for NVIDIA cards (4GB recommended for future models).
• Links:
  • RWKV-LM training code: https://github.com/BlinkDL/RWKV-LM
  • Web version: https://blinkdl.github.io/AI-Writer/
  • ModelScope online version: https://modelscope.cn/studios/BlinkDL/RWKV-CHN/summary

Highlighted Details

• Supports NVIDIA and AMD/Intel GPU acceleration.
• RWKV model architecture offers efficient inference compared to traditional Transformers.
• Special sampling methods are used to improve generation quality for smaller models.
• Includes a web interface for easier use.

Maintenance & Community

• The project README indicates that the current models and information are outdated, recommending the newer RWKV-Runner project.
• QQ groups are available for support and discussion (553456870, 325154699).
• Telegram group: https://t.me/ai_writer

Licensing & Compatibility

• The README does not explicitly state a license. The associated RWKV-LM repository uses the Apache 2.0 license. Compatibility for commercial use is not specified.

Limitations & Caveats

The project explicitly states that training data is derived solely from web novels, leading to a lack of common sense in generated text, and that outputs are for entertainment purposes only. The README also notes that the current models are outdated.