This project provides a generic, easy-to-use serving service for machine learning models, primarily targeting developers and researchers who need to deploy models via RESTful APIs. It aims to simplify the deployment process, offering broad framework support and client generation capabilities.

How It Works

The service utilizes Flask to start an HTTP server and loads TensorFlow models using tf.saved_model.loader. It constructs feed_dict from incoming JSON requests, executes inference via sess.run(), and supports multiple model versions through independent threads. Client code generation is handled by reading user models and rendering templates with Jinja.

Quick Start & Requirements

• Install: pip install simple_tensorflow_serving
• Prerequisites: Python, TensorFlow. GPU support requires CUDA.
• Docker: Available for CPU, GPU, HDFS, and Python 3.4.
• Docs: https://stfs.readthedocs.io/

Highlighted Details

• Supports TensorFlow, MXNet, PyTorch, ONNX, Scikit-learn, XGBoost, PMML, and more.
• Offers GPU acceleration and dynamic loading of custom TensorFlow ops.
• Generates clients in various languages (Python, Go, JavaScript, etc.) from models.
• Provides basic authentication and TLS/SSL support.
• Handles image models via raw file uploads.

Maintenance & Community

The project is hosted on GitHub and welcomes contributions via issues and pull requests.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

While claiming similar performance to TensorFlow Serving, benchmarks indicate TensorFlow Serving performs better, especially with GPUs. The project does not specify its maintenance status or active development.