Generative Chat Bot Implementation Using Deep Recurrent Neural Networks and Natural Language Understanding

7 Pages Posted: 29 Mar 2019

See all articles by Niranjan Zalake

Niranjan Zalake

SVKM's Narsee Monjee Institute of Management Studies (NMIMS)

Gautam Naik

Tata Consultancy Services (TCS)

Date Written: March 29, 2019

Abstract

There has been not much development in the area of neural conversational models/dialogue systems till the recent times. Neural networks are gaining much more importance once again due to the exponentially decreasing cost of memory and cheap cloud services which has made it possible to do such huge computations with ease. In this paper, we present an architecture of recurrent neural network called as Sequence to Sequence model which is unlike traditional dialogue systems built until now. The architecture aims at building the neural network without using components like Named Entity Recognition (NER) and huge lines of code with conditional statements to be written to get decent performance. It actually consists of two neural networks, encoder-decoder. The encoder encodes input sequence of tokens into a neural machine readable form and decoder decodes the sequence output from encoder. The architecture is complemented with the attention mechanism which allows to pay attention to certain parts of the input sequence which are more important in generating output sequence. In this paper, we also show that using the Bidirectional Long Short Term Memory (LSTM) cells instead of regular RNN cells or GRU's, increases the performance in terms of model convergence and performance. Using this approach we aim to deliver a conversational model with performance same as the current one with very less overhead. We have selected an open domain as the target as it is necessary to get dialogues of a particular domain to get optimum performance from the model.

Keywords: Recurrent Neural Network, Long Short Term Memory, Attention Mechanism, Beam Search, BLEU Score, Deep Learning, Bidirectional RNN, Chatbot, Generative bots, Natural Language Understanding

Suggested Citation

Zalake, Niranjan and Naik, Gautam, Generative Chat Bot Implementation Using Deep Recurrent Neural Networks and Natural Language Understanding (March 29, 2019). Proceedings 2019: Conference on Technologies for Future Cities (CTFC), Available at SSRN: https://ssrn.com/abstract=3362123 or http://dx.doi.org/10.2139/ssrn.3362123

Niranjan Zalake (Contact Author)

SVKM's Narsee Monjee Institute of Management Studies (NMIMS) ( email )

V. L. Mehta Road
Vile Parle (W)
Mumbai, Maharashtra 400056
India

Gautam Naik

Tata Consultancy Services (TCS) ( email )

IDC, Akruti Business Port Road No. 13, Mandheri
Hi Tec City, Madhapur
Mumbai, Maharashtra 400079
India

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
427
Abstract Views
1,967
Rank
125,447
PlumX Metrics