keras_lr_finder  by surmenok

Keras tool plots loss change during learning rate increase

created 7 years ago
258 stars

Top 98.6% on sourcepulse

GitHubView on GitHub
Project Summary

This library helps Keras users find an optimal learning rate for their deep neural networks by plotting loss function behavior against exponentially increasing learning rates. It's designed for researchers and practitioners seeking to improve model training stability and convergence.

How It Works

The LRFinder class integrates with Keras models. It trains the model for a specified number of epochs with a learning rate that exponentially increases from a defined start to end value. During this process, it records the loss for each learning rate step. This data is then used to generate plots visualizing the loss and its rate of change, allowing users to identify a suitable learning rate range.

Quick Start & Requirements

  • Install via pip: pip install keras-lr-finder
  • Requires TensorFlow/Keras.
  • Usage examples are provided in the README.

Highlighted Details

  • Plots loss vs. learning rate.
  • Plots rate of change of loss vs. learning rate.
  • Supports skipping initial and final batches for cleaner plots.
  • Based on Leslie N. Smith's "Cyclical Learning Rates" paper and fastai implementation.

Maintenance & Community

Contributions are welcome via GitHub issues and pull requests.

Licensing & Compatibility

The license is not explicitly stated in the README. Compatibility with commercial or closed-source projects is unclear without a specified license.

Limitations & Caveats

The README does not specify a license, which may impact commercial use. The library is presented as a standalone tool without explicit versioning or release notes, suggesting potential for breaking changes.

Health Check
Last commit

2 months ago

Responsiveness

1 week

Pull Requests (30d)
0
Issues (30d)
0
Star History
3 stars in the last 90 days

Explore Similar Projects

Feedback? Help us improve.