Systems and methods for end-to-end handwritten text recognition using neural networks

A processor implemented method for end-to-end handwritten text recognition using neural networks, the method comprising the steps of:

• pre-processing, by an image pre-processing module, a received scanned handwritten text input image, wherein the pre-processing comprises:
• obtaining one or more sub-images from the received scanned handwritten text input image, wherein each sub-image comprises a line of handwritten text present in the scanned handwritten text input image;
• and
• scaling the one or more sub-images into one or more scaled sub-images having a fixed height and a variable width and maintaining aspect ratio thereof;
• transforming, by a feature extraction module, each of the one or more scaled sub-images into one or more feature maps by applying a Convolutional Neural Network (CNN), wherein the one or more feature maps contain visual features of a corresponding sub-image;
• converting, by a map-to-sequence module, the one or more feature maps into a sequence of feature vectors by detaching depth-wise, columns comprised in the one or more feature maps and concatenating respective columns of the one or more feature maps of the corresponding sub-image;
• generating, by a sequence learning module, one or more set of characters, by mapping the visual features associated with each sequence of feature vectors, using a recurrent neural network (RNN) based encoder unit and decoder unit, wherein the one or more set of characters is a machine readable text recognized for the corresponding sub-image, and wherein an encoder unit of the RNN based encoder unit and decoder unit is configured to generate a context vector for each sequence of feature vectors and a decoder unit of the RNN based encoder unit and decoder unit is configured to apply a conditional probability distribution over the context vector for each sequence of feature vectors, to generate a set of characters for the corresponding sub-image;
• and
• post-processing, by an image post-processing module, the set of characters, wherein the post-processing comprises combining the set of characters for each of the corresponding sub-image to generate a handwritten text output file, wherein the hand-written text output file is a recognized text version of the scanned handwritten text input image.

A non-transitory computer readable medium storing a computer readable program embodied therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

• pre-process a received scanned handwritten text input image by an image pre-processing module, wherein the pre-process comprises:
• obtaining one or more sub-images from the received scanned handwritten text input image, wherein each sub-image comprises a line of handwritten text present in the scanned handwritten text input image;
• and
• scaling the one or more sub-images into one or more scaled sub-images having a fixed height and a variable width and maintaining aspect ratio thereof;
• transform each of the one or more scaled sub-images by a feature extraction module, into one or more feature maps by applying a Convolutional Neural Network (CNN), wherein the one or more feature maps contain visual features of a corresponding sub-image;
• convert the one or more feature maps by a map-to-sequence module, into a sequence of feature vectors by detaching depth-wise, columns comprised in the one or more feature maps and concatenating respective columns of the one or more feature maps of the corresponding sub-image;
• generate one or more set of characters by a sequence learning module, by mapping the visual features associated with each sequence of feature vectors, using a recurrent neural network (RNN) based encoder unit and decoder unit, wherein the one or more set of characters is a machine readable text recognized for the corresponding sub-image, and wherein an encoder unit of the RNN based encoder unit and decoder unit is configured to generate a context vector for each sequence of feature vectors and a decoder unit of the RNN based encoder unit and decoder unit is configured to apply a conditional probability distribution over the context vector for each sequence of feature vectors, to generate a set of characters for the corresponding sub-image;
• and
• post-process the set of characters by an image post-processing module, wherein the post-process comprises combining the set of characters for each of the corresponding sub-image to generate a handwritten text output file, wherein the hand-written text output file is a recognized text version of the scanned handwritten text input image.