BCNet addresses the challenge of instance segmentation in scenes with occluded objects by explicitly modeling occlusion relationships. It targets researchers and practitioners in computer vision seeking state-of-the-art performance in complex scenes, offering improved accuracy through its novel bilayer decoupling approach.

How It Works

BCNet introduces a novel mask head that models image formation as a composition of two overlapping layers: an occluder layer and an occludee layer. This "bilayer decouple" approach explicitly separates the object boundary and mask predictions for both occluding and occluded instances within the same region of interest. This allows for more accurate segmentation of overlapping objects by considering their interaction and disentangling occluder and occludee boundaries, leading to improved performance on standard detectors like Faster R-CNN and FCOS.

Quick Start & Requirements

• Install: Requires conda for environment setup, followed by pip install for dependencies and the BCNet package.
• Prerequisites: PyTorch 1.4.0, torchvision 0.5.0, CUDA toolkit 10.1, Python 3.7, ninja, yacs, cython, matplotlib, tqdm, opencv-python==4.4.0.40, scikit-image, and pycocotools.
• Dataset: COCO 2017 dataset with converted mask annotations for bilayer decoupling training.
• Resources: Multi-GPU training is supported.
• Links: Official Implementation, Paper, COCO-OCC Split

Highlighted Details

• Achieves state-of-the-art performance on COCO test-dev, with mAP(mask) scores up to 41.2 (FCOS, Res-X101 FPN).
• Explicitly models occlusion using a bilayer structure with two GCN layers for occluder and occludee.
• Visualizations show the network's ability to handle multiple occluders within a single ROI.
• Integrates seamlessly with both anchor-based (Faster R-CNN) and anchor-free (FCOS) detectors.

Maintenance & Community

• Developed by Lei Ke, Yu-Wing Tai, and Chi-Keung Tang (CVPR 2021).
• Related works include Mask Transfiner (CVPR 2022), VOIN (ICCV 2021).
• Contact: lkeab@cse.ust.hk or GitHub issues.

Licensing & Compatibility

• MIT License.
• Compatible with commercial use and closed-source linking.

Limitations & Caveats

• Requires specific older versions of PyTorch (1.4.0) and OpenCV (4.4.0.40), which may pose compatibility challenges with newer environments.
• The installation process involves manual cloning and building of pycocotools and requires specific dataset annotation conversions.