This is built chatbot using deep learning with PyTorch and the also by using some concept of Natural Language Processing like Tokenization, Stemming and Bag of words and used the library nltk.
Approach and Concepts used
The Natural Language Toolkit (nltk) was used which is a Natural Language processing library. The concept of natural language processing used were Tokenization, Stemming and bag of words.
Tokenization is splitting a string into meaningful units. Eg. words punctuations characters, numbers.
For example - “Hey there ! It’s 3 pm”
this will get converted to this —
[“Hey”, “there”, “!”, “It”, “‘s”, “3”, “pm”]
Stemming is basically it generates the root form of the words. Crude heuristic that chops of the ends of the words.
For example - “organize”, “organizes”, “organizing”
this will get converted to this
[“organ”, “organ”, “organ”]
So afterwards this tokenization and stemming is applied on all the patterns. So a large array is formed having all the words in the pattern know as bag of words. This bag of words has all the word contained in it. This bag of word is send to the neural networks layers which tells is and see if the word entered is there or not in the array.
Files in this project
Intent.json - To get the data for training
nltk_util - For applying various NLP concepts
Train - For training the nn model
Model - For building a nn model
Chat - The implementation of the ChatBot