This repository provides Seal, a framework for segmenting automotive point cloud sequences by distilling knowledge from Vision Foundation Models (VFMs). It targets researchers and engineers working with 3D perception, offering a self-supervised approach that leverages spatial and temporal consistency without requiring manual annotations during pretraining.

How It Works

Seal distills knowledge from VFMs into point clouds by generating semantic superpixels and superpoints. It enforces spatial consistency between LiDAR and camera features and temporal consistency between segments across frames. This cross-modal learning approach enables effective knowledge transfer to diverse point cloud datasets.

Quick Start & Requirements

• Installation details are in INSTALL.md.
• Data preparation instructions are in DATA_PREPARE.md.
• Superpoint generation details are in SUPERPOINT.md.
• Usage examples are in GET_STARTED.md.
• Requires PyTorch and MMDetection3D.

Highlighted Details

• Achieved state-of-the-art performance on benchmarks like nuScenes, KITTI, and Waymo Open Dataset.
• Demonstrates strong generalization across various point cloud types (real/synthetic, low/high-resolution, clean/corrupted).
• Exhibits robustness to sensor noise and adverse weather conditions.
• Offers video demos and multiple live demos for qualitative assessment.

Maintenance & Community

• The project was a spotlight at NeurIPS 2023.
• Code released in July 2023, with the paper available on arXiv.
• Built upon the MMDetection3D codebase and adapted from several other open-source repositories.

Licensing & Compatibility

• Licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
• Non-commercial use is permitted, but commercial use or linking with closed-source projects may be restricted by the license.

Limitations & Caveats

The current release focuses on automotive point clouds and may require adaptation for other domains. While pretraining is self-supervised, downstream tasks may require fine-tuning with labeled data. The "TODO List" indicates that evaluation and training details are still to be added.