Sign Language Recognition Using ResNet50 Deep Neural Network Architecture

7 Pages Posted: 27 Feb 2020

See all articles by Pulkit Rathi

Pulkit Rathi

ABV - Indian Institute of Information Technology and Management

Raj Kuwar Gupta

ABV - Indian Institute of Information Technology and Management

Soumya Agarwal

ABV-IIITM

Anupam Shukla

Indian Institute of Information Technology (IIIT), Pune

Date Written: February 27, 2020

Abstract

Communication is a barrier between the deaf-mute community and the rest of the society. Sign Language is used for communication among such people who cannot speak and listen. The automation of sign language recognition has gained researchers attention in the last few years. Many complex and costly hardware systems have already been developed to assist the purpose. However, we propose to use deep learning approach for automated sign language recognition. We devised a novel 2-level ResNet50 based Deep Neural Network Architecture to classify fingerspelled words. The dataset used is the standard American Sign Language Hand gesture dataset by [1]. The dataset was first augmented using various augmentation techniques. In our 2-level ResNet50 based approach the Level 1 model classifies the input image into one of the 4 sets. After an image is classified into one of the sets it is provided as an input to the corresponding second level model for predicting the actual class of the image. Our approach yields an accuracy of 99.03% on 12,048 test images.

Keywords: sign language recognition, gesture classification, convolution neural network, transfer learning resnet50

Suggested Citation

Rathi, Pulkit and Kuwar Gupta, Raj and Agarwal, Soumya and Shukla, Anupam, Sign Language Recognition Using ResNet50 Deep Neural Network Architecture (February 27, 2020). 5th International Conference on Next Generation Computing Technologies (NGCT-2019). Available at SSRN: https://ssrn.com/abstract=3545064 or http://dx.doi.org/10.2139/ssrn.3545064

Pulkit Rathi (Contact Author)

ABV - Indian Institute of Information Technology and Management ( email )

Gwalior

Raj Kuwar Gupta

ABV - Indian Institute of Information Technology and Management ( email )

Gwalior

Soumya Agarwal

ABV-IIITM ( email )

18, Geeta Colony, Dal Bazar
Lashkar
Gwalior, Madhya Pradesh 474004
India

Anupam Shukla

Indian Institute of Information Technology (IIIT), Pune ( email )

Pune, Maharashtra 411048
India

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
42
Abstract Views
166
PlumX Metrics