This repository provides a unified benchmark for parameter-efficient transfer learning (PETL) in computer vision, addressing the challenge of inconsistent evaluation across numerous PETL methods. It targets researchers and practitioners in computer vision who need to compare and select PETL techniques for various downstream tasks, offering a standardized framework and codebase for fair evaluation.

How It Works

The V-PETL Bench standardizes the evaluation of 25 PETL algorithms across 30 diverse computer vision datasets, covering image recognition, video action recognition, and dense prediction. It employs a modular and extensible codebase, allowing for systematic comparison of methods like Adapter, LoRA, and VPT, while providing detailed results and performance metrics against full fine-tuning and linear probing baselines.

Quick Start & Requirements

• Install: Clone the repository and install dependencies via pip install -r requirements.txt within a Python 3.8+ conda environment.
• Prerequisites: PyTorch, torchvision, torchaudio, timm. Pre-trained model checkpoints (e.g., ViT-B/16, Swin-B) are required and links are provided. Datasets need to be downloaded separately and placed in specified directories.
• Setup: Requires downloading multiple datasets and pre-trained models, which can be time-consuming.
• Documentation: Homepage, Paper, Document

Highlighted Details

• Evaluates 25 PETL algorithms on 30 diverse CV datasets.
• Provides comprehensive benchmark results for image classification, video action recognition, and dense prediction tasks.
• Includes a modular and extensible codebase for easy integration of new PETL methods.
• Offers detailed performance comparisons, including parameter efficiency (PPT) metrics.

Maintenance & Community

The project is maintained by Yi Xin and Siqi Luo from Nanjing University and Shanghai Jiao Tong University, respectively.

Licensing & Compatibility

The repository appears to be under a permissive license, but specific details are not explicitly stated in the README. Compatibility for commercial use or closed-source linking would require verification of the underlying dependencies and any specific license terms for the datasets or pre-trained models.

Limitations & Caveats

The README indicates that checkpoints for dense prediction tasks on COCO and PASCAL VOC are "Coming Soon," suggesting incomplete benchmark results for these specific areas. Dataset preparation involves manual downloading and organization, which can be a significant hurdle.