Generative Chat Bot Implementation Using Deep Recurrent Neural Networks and Natural Language Understanding
7 Pages Posted: 29 Mar 2019
Date Written: March 29, 2019
There has been not much development in the area of neural conversational models/dialogue systems till the recent times. Neural networks are gaining much more importance once again due to the exponentially decreasing cost of memory and cheap cloud services which has made it possible to do such huge computations with ease. In this paper, we present an architecture of recurrent neural network called as Sequence to Sequence model which is unlike traditional dialogue systems built until now. The architecture aims at building the neural network without using components like Named Entity Recognition (NER) and huge lines of code with conditional statements to be written to get decent performance. It actually consists of two neural networks, encoder-decoder. The encoder encodes input sequence of tokens into a neural machine readable form and decoder decodes the sequence output from encoder. The architecture is complemented with the attention mechanism which allows to pay attention to certain parts of the input sequence which are more important in generating output sequence. In this paper, we also show that using the Bidirectional Long Short Term Memory (LSTM) cells instead of regular RNN cells or GRU's, increases the performance in terms of model convergence and performance. Using this approach we aim to deliver a conversational model with performance same as the current one with very less overhead. We have selected an open domain as the target as it is necessary to get dialogues of a particular domain to get optimum performance from the model.
Keywords: Recurrent Neural Network, Long Short Term Memory, Attention Mechanism, Beam Search, BLEU Score, Deep Learning, Bidirectional RNN, Chatbot, Generative bots, Natural Language Understanding
Suggested Citation: Suggested Citation