This repository provides the official code and datasets for the paper "Language Models Represent Space and Time," enabling researchers to extract spatial and temporal world models from Large Language Models (LLMs). It is primarily aimed at researchers in AI, cognitive science, and linguistics interested in understanding LLM capabilities beyond text generation. The project offers cleaned datasets for analyzing how LLMs encode world knowledge.

How It Works

The project focuses on probing LLMs to understand their internal representations of space and time. It involves extracting entity names and associated metadata from LLMs, tokenizing this data for specific models like Llama and Pythia, and providing experimental infrastructure for analysis. This approach allows for a systematic investigation into the structured knowledge LLMs can acquire.

Quick Start & Requirements

• Datasets are available in data/entity_datasets/ and tokenized versions in data/prompt_datasets/.
• A minimal version of the code for basic probing experiments will be released soon.
• Requires Python and potentially specific LLM libraries for running experiments.

Highlighted Details

• Focuses on extracting spatial and temporal world models from LLMs.
• Provides cleaned entity datasets and tokenized versions for Llama and Pythia.
• Contains experimental infrastructure for the paper "Language Models Represent Space and Time."

Maintenance & Community

The project is associated with Wes Gurnee and Max Tegmark. Further community engagement details are not provided in the README.

Licensing & Compatibility

The repository's license is not specified in the README.

Limitations & Caveats

The full experimental code is not yet released, with only datasets currently available. The project is primarily research-oriented, and its direct applicability for production systems is not detailed.