This project provides a Python script to accelerate downloads from Hugging Face, specifically targeting users in regions with slower access to the official Hugging Face servers. It leverages the huggingface-cli and hf_transfer tools to download models and datasets from mirror sites, offering a significant speed improvement for domestic users.

How It Works

The script acts as a wrapper around huggingface-cli and hf_transfer. It defaults to using the hf-mirror.com mirror site and the hf_transfer library for accelerated downloads. Users can specify models or datasets by name and optionally direct downloads to a specific directory. The script also includes options to include or exclude specific files during the download process.

Quick Start & Requirements

• Install and run via: python hf_download.py --model <model_name> --save_dir <path>
• Prerequisites: Python, huggingface-cli, hf_transfer (automatically installed via pip).
• Additional requirements: Hugging Face access token for gated models.
• Setup: Minimal, relies on pip for dependency installation.
• Documentation: Official Hugging Face CLI documentation

Highlighted Details

• Utilizes hf-transfer for potentially faster downloads.
• Supports downloading from mirror sites (hf-mirror.com by default) for improved speeds in certain regions.
• Includes --include and --exclude flags for granular file selection during downloads.
• Provides guidance on managing the Hugging Face cache directory via soft links for flexible storage.

Maintenance & Community

The project appears to be a personal utility script with no explicit mention of a dedicated community or active maintenance beyond the initial author's updates.

Licensing & Compatibility

The repository does not explicitly state a license. It relies on Hugging Face's official tools, which are typically under permissive licenses. Compatibility for commercial use would depend on the licenses of huggingface-cli and hf_transfer.

Limitations & Caveats

The script is a simple wrapper and may not support all advanced features of huggingface-cli. Users needing more complex functionality should refer to the official documentation. The effectiveness of the acceleration depends on the mirror site's performance and network conditions.