Loglizer is a machine learning toolkit for automated anomaly detection in system logs, targeting researchers and engineers. It provides a framework for parsing logs, extracting features, and applying various supervised and unsupervised models to identify abnormal system behavior.

How It Works

Loglizer implements a standard log analysis pipeline: log parsing to structure unstructured messages, feature extraction (e.g., event counting vectors) using windowing techniques, and anomaly detection via machine learning models. This approach allows for the application of established ML algorithms to log data for robust anomaly detection.

Quick Start & Requirements

• Install via pip install -r requirements.txt after cloning the repository.
• Requires Python. Specific version not stated.
• Demo available in docs.

Highlighted Details

• Implements supervised (LR, Decision Tree, SVM) and unsupervised (LOF, One-Class SVM, Isolation Forest, PCA, Invariants Mining, Clustering) anomaly detection models.
• Includes benchmarking results for HDFS dataset, showing high precision and F1 scores for models like Decision Tree and SVM.
• Provides access to labeled log datasets via the loghub project.

Maintenance & Community

• Developed by researchers from The Chinese University of Hong Kong.
• Feedback channel via GitHub issues.
• Repository has seen significant code rewrites and restructuring.

Licensing & Compatibility

• License not explicitly stated in the README.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The README notes that ML models require parameter tuning for optimal performance on custom data. The specific Python version requirement is not detailed, and licensing for commercial use is unclear.