nmt-chatbot  by daniel-kukiela

NMT chatbot using seq2seq for sentence translation

created 7 years ago
389 stars

Top 74.9% on sourcepulse

GitHubView on GitHub
Project Summary

This project implements a Neural Machine Translation (NMT) chatbot, offering sentence translation capabilities alongside its primary chatbot function. It targets developers and researchers interested in building conversational AI using seq2seq models, providing a custom BPE/WPM-like tokenizer and an embedded detokenizer for improved vocabulary management and translation quality.

How It Works

The chatbot leverages a seq2seq architecture built upon a fork of the NMT framework. It features a custom implementation of BPE/WPM-like subword tokenization, which allows for a smaller vocabulary size by breaking down rare words into common subword units. This approach aims to reduce <unk> tokens and improve translation accuracy. An embedded detokenizer is used, simplifying the detokenization process by avoiding rule-based systems.

Quick Start & Requirements

  • Install via pip install -r requirements.txt after cloning the repository.
  • Requires Python 3.6+, TensorFlow-GPU, CUDA Toolkit 8.0, and cuDNN 6.1.
  • Training data should be placed in the new_data folder.
  • Official tutorials: Windows, Linux

Highlighted Details

  • Implements both standard and BPE/WPM-like subword tokenization.
  • Includes rule-based systems for synonym replacement and answer scoring.
  • Offers an interactive command-line interface for inference and batch processing.
  • Supports importing the chatbot as a module into other Python projects.

Maintenance & Community

The project is a fork of NMT, with modifications to support TensorFlow 1.4. Recent updates include a new response scoring engine, performance improvements for data preparation, and relative model paths for easier deployment.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project requires specific, older versions of TensorFlow, CUDA, and cuDNN, which may pose compatibility challenges with modern hardware and software stacks. The fork nature means maintaining synchronization with upstream NMT changes could be a burden.

Health Check
Last commit

5 years ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
4 stars in the last 90 days

Explore Similar Projects

Feedback? Help us improve.