Bonnet is an open-source framework for training and deploying Convolutional Neural Networks (CNNs) for semantic segmentation in robotics. It targets researchers and engineers working with robotic systems, offering a unified pipeline for both model development and efficient deployment on embedded platforms. The framework aims to simplify the process of integrating deep learning for perception tasks in robotics.

How It Works

Bonnet utilizes a Python-based training pipeline with TensorFlow and OpenCV for data augmentation and model building. For deployment, it offers C++ applications that can run standalone or as ROS nodes. The C++ backend is designed for extensibility, with current support for TensorFlow and TensorRT, prioritizing performance on NVIDIA GPUs and embedded systems like Jetson. This dual approach allows leveraging Python's flexibility during training and C++'s efficiency for real-time inference.

Quick Start & Requirements

• Installation: A Docker image is provided: docker pull tano297/bonnet:cuda9-cudnn7-tf17-trt304.
• Prerequisites: CUDA 9, cuDNN 7, TensorFlow 1.7, TensorRT 3.0.4. X11 forwarding is needed for GUI display.
• Resources: Tested on x86 Ubuntu with NVIDIA GPUs (GeForce 940MX, GTX1080Ti) and Jetson TX2. CPU-only inference is also supported.
• Documentation: Links to training and deployment READMEs are available within the main README.

Highlighted Details

• Supports multiple datasets including Cityscapes, Synthia, and custom datasets.
• Offers pre-trained models for various resolutions and tasks like person segmentation and crop vs. weed segmentation.
• Optimized for performance on NVIDIA hardware via TensorRT, enabling fast inference.
• Includes ROS nodes for seamless integration into robotic systems.

Maintenance & Community

The project is primarily maintained by Andres Milioto from the University of Bonn. The README mentions contributions and beta testing by other individuals. There are no explicit links to community channels like Discord or Slack.

Licensing & Compatibility

Bonnet is licensed under the GNU General Public License v3.0 or later. This is a copyleft license, meaning derivative works must also be made available under the GPL. Pre-trained models retain the copyright of their respective datasets.

Limitations & Caveats

The framework is built around TensorFlow 1.7 and CUDA 9/cuDNN 7, which are older versions. The project lists several "TODOs" including adding Movidius Neural Stick backends and multi-camera ROS nodes, indicating ongoing development but also potential missing features for certain platforms.