This repository provides a TensorFlow implementation of UnFlow, a method for unsupervised learning of optical flow using a bidirectional census loss. It is targeted at researchers and practitioners in computer vision and deep learning who need to compute optical flow without ground truth data. The primary benefit is enabling robust optical flow estimation in scenarios where ground truth is unavailable or expensive to obtain.

How It Works

UnFlow trains deep networks end-to-end for dense optical flow estimation without requiring ground truth flow. It utilizes a novel unsupervised proxy loss, specifically a bidirectional census loss, which enforces consistency between forward and backward flow predictions. This approach allows the model to learn meaningful flow representations from readily available video sequences.

Quick Start & Requirements

• Install: Clone the repository and install dependencies via pip.
• Prerequisites: Python 3, GCC4, RAR backend tool, tensorflow-gpu (>= 1.7), CUDA, CuDNN. Requires NVIDIA GPU (8GB+ recommended, 11-12GB for stacked variants).
• Setup: Modify config_template/config.ini and run experiments using python run.py --ex <experiment_name>.
• Data: Datasets (SYNTHIA, KITTI, FlyingChairs) are downloaded on-demand. Cityscapes requires manual download. Ensure ~150GB of disk space.
• Links: Slides, TensorFlow Releases for compatibility.

Highlighted Details

• Supports unsupervised training on SYNTHIA, KITTI raw, Cityscapes, and FlyingChairs.
• Enables supervised fine-tuning on KITTI 2012/2015 flow datasets.
• Evaluates on KITTI 2012/2015, Middlebury, and MPI Sintel flow benchmarks.
• Implements FlowNetS, FlowNetC, and stacked variants from FlowNet 2.0.

Maintenance & Community

The project was released in 2018. The author notes potential compatibility issues with newer TensorFlow versions due to unstable custom op compilation and is currently too busy to update the code, welcoming contributions.

Licensing & Compatibility

Released under the MIT License, permitting commercial use and closed-source linking.

Limitations & Caveats

The project may have compatibility issues with recent TensorFlow versions due to custom op compilation. The author is seeking contributions to address these issues. Training can be time-consuming, especially with limited GPU memory.