This project provides a continuous-action simulation environment for landing a SpaceX Falcon 9 rocket, built using Python and OpenAI Gym with the Box2D physics engine. It targets researchers and engineers interested in comparing classical control methods (PID, MPC, LQR) with AI algorithms (DDPG, ES, FA Q-Learning) for continuous control problems, offering a more realistic scenario than discrete-action simulators.

How It Works

The simulator models a vertical rocket landing, focusing on a continuous action space for thruster control (main engine, side thrusters, nozzle angle). This contrasts with discrete action spaces found in similar environments like LunarLander. The core advantage is enabling the evaluation of advanced control strategies like Deep Deterministic Policy Gradients (DDPG), which uses an actor-critic architecture with neural networks to handle continuous states and actions effectively.

Quick Start & Requirements

• Run simulation: python main_simulation.py
• Prerequisites: tensorflow, matplotlib, gym, numpy, Box2D, logging, pyglet, cvxpy. Windows users may need specific extension libraries for cvxpy.
• Setup time: Not specified, but cvxpy installation can be time-consuming.

Highlighted Details

• Compares PID, MPC, ES, and DDPG controllers.
• DDPG demonstrated impressive results, outperforming other methods.
• State defined as: [x_pos, y_pos, x_vel, y_vel, lateral_angle, angular_velocity].
• Actions include main engine thrust, side thrusters, and nozzle angle.

Maintenance & Community

• Version 1.1.0.
• Authors: Reuben Ferrante.
• No community links (Discord/Slack) or roadmap are provided in the README.

Licensing & Compatibility

• MIT License.
• Permissive for commercial use and closed-source linking.

Limitations & Caveats

The README mentions "unstructured scripts" and "messy untested code" in the controller implementations, suggesting potential quality issues outside the core environment. Modern Predictive Control (MPC) is noted as "not generalized."