This repository provides a PyTorch-based codebase for state-of-the-art video understanding tasks, targeting researchers and engineers. It simplifies the implementation and evaluation of various video classification models, offering a high-performance, modular design for rapid experimentation with novel research ideas.

How It Works

X-Temporal supports multiple input formats, including raw videos, RGB frames, and optical flow frames, and is designed to handle both single-label and multi-label datasets. Its modular architecture allows for easy integration and comparison of popular video understanding frameworks like SlowFast, R(2+1)D, R3D, TSN, and TSM.

Quick Start & Requirements

• Install: Clone the repository and run ./easy_setup.sh.
• Prerequisites: PyTorch 1.0+, TensorboardX, tqdm, scikit-learn, decord. FFmpeg is recommended for frame extraction.
• Data Format: Supports meta files with video path, frame count, and category ID, or direct reading of original video files using decord. Multi-label datasets require categories separated by commas.
• Resources: Setup involves cloning and running a script; training and testing require GPU resources and dataset preparation.
• Links: Challenge Website

Highlighted Details

• Implements SOTA video understanding methods including TSN, TIN, TSM, R(2+1)D, R3D, and SlowFast.
• Supports popular datasets like Kinetics, Something2Something, and Multi-Moments in Time.
• Achieved 1st place in the ICCV19-Multi Moments in Time Challenge.
• Offers flexibility in input data types (raw video, frames, flow) and label types (single/multi-label).

Maintenance & Community

Maintained by Hao Shao, ManYuan Zhang, and Yu Liu. The project was released in August 2020.

Licensing & Compatibility

Released under the MIT license, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

The codebase was last updated in August 2020, and its compatibility with the latest PyTorch versions or newer SOTA models is not guaranteed.