This repository provides code and results for GPT-NER, a method for Named Entity Recognition (NER) leveraging Large Language Models (LLMs). It targets researchers and practitioners in NLP seeking to apply LLMs to NER tasks, offering a framework for few-shot and zero-shot NER with GPT-3, demonstrating competitive performance against supervised baselines on standard datasets.

How It Works

GPT-NER utilizes GPT-3 for NER by framing the task as a generation problem. It explores different retrieval strategies for providing context to the LLM: random retrieval, sentence-level embeddings (using SimCSE), and entity-level embeddings. The approach aims to enhance NER performance by effectively guiding the LLM with relevant examples, particularly in few-shot scenarios.

Quick Start & Requirements

• Install: pip install openai==0.27.2 simcse==0.4
• Prerequisites: Python >= 3.7.3, OpenAI API key (set as environment variable OPENAI_API_KEY).
• Data: MRC-NER dataset (full) or sampled 100-dataset (Google Drive link provided).
• SimCSE Model: sup-simcse-roberta-large (link provided).
• Usage: Scripts are located in openai_access/scripts/. Refer to openai_access/get_results_mrc_knn.py and openai_access/verify_results.py for argument details.
• Evaluation: Use openai_access/scripts/compute_f1.sh.

Highlighted Details

• Demonstrates GPT-3 performance on Flat NER (CoNLL2003, OntoNotes5.0) and Nested NER (ACE2004, ACE2005, GENIA) datasets.
• Achieves competitive results, particularly with entity-level embedding retrieval, outperforming some supervised methods on sampled data.
• Includes self-verification scripts for zero-shot and few-shot evaluations.
• Code is structured around OpenAI API access and SimCSE for embeddings.

Maintenance & Community

• Primary contact: wangshuhe@stu.pku.edu.cn.
• The project is associated with a research paper published on arXiv.

Licensing & Compatibility

• The repository itself does not explicitly state a license.
• Usage of OpenAI's GPT-3 requires adherence to OpenAI's terms of service and API usage policies.

Limitations & Caveats

• Reliance on the OpenAI API means costs are associated with usage, and access is subject to OpenAI's availability and policies.
• Performance is heavily dependent on the quality of retrieved examples and the chosen embedding strategy.
• The README mentions that accessing GPT-3 can be expensive, advising users to start with the sampled dataset.