This repository provides code and datasets for P-tuning, a novel method for parameter-efficient tuning of large language models. It is targeted at researchers and practitioners looking to adapt LLMs with reduced computational resources.

How It Works

P-tuning introduces a continuous, trainable prompt that is prepended to the input sequence. This prompt is learned via gradient descent, allowing the model to adapt to specific tasks without fine-tuning all of its parameters. This approach offers a more efficient alternative to full model fine-tuning, requiring fewer trainable parameters and potentially less data.

Quick Start & Requirements

• Installation: Follow instructions in README.md within subdirectories for LAMA and few-shot SuperGLUE experiments.
• Datasets: LAMA dataset should be placed in ./data, and SuperGLUE dataset in the project root.
• Dependencies: Refer to requirement.txt in relevant subdirectories.

Highlighted Details

• P-tuning v2 is available in a separate GitHub repository.
• Associated work includes GLM-130B, an open bilingual LLM.
• Experiments are available for LAMA and few-shot SuperGLUE benchmarks.

Maintenance & Community

The project is associated with THUDM (Tsinghua University Knowledge Engineering Group).

Licensing & Compatibility

The README does not explicitly state the license.

Limitations & Caveats

The provided code and datasets are specifically for the experiments detailed in the paper "GPT Understands, Too". Compatibility with other models or tasks may require adaptation.