Systems and methods of instant-messaging bot for robotic process automation and robotic textual-content extraction from images

A system of instant-messaging bot for robotic process automation comprising:

• a chatbot application for an enterprise (or organization) including software for receiving, processing, analyzing and responding to human-generated messages in a human-like manner;
• a software RPA manager for said enterprise containing multiple workflow modules and receiving instructions from said chatbot for executing said workflows;
• an instant-messaging (IM) platform for said enterprise connected to said chatbot application and separately connected to one or more public IM platforms via the Internet;
• and
• said enterprise IM platform including software for managing traffic of IM messages exchanged among said chatbot, users of said enterprise connected to said enterprise IM platform, and external users connected to said public IM platforms;
• wherein:
• said workflow module is for textual-content extraction from pixelated digital images provided by said enterprise users via said enterprise IM platform, or by said external users via said public IM platforms;
• said textual-content extraction comprises:
• a text-detection step for determining locations and sizes of text lines in said images, followed by a text-recognition step for recognizing textual content of each detected text line, and
• said text-detection step and text-recognition steps being accomplished by convolutional neural network (CNN) methods;
• said text-recognition method comprises:
• a convolutional neural network (CNN) backbone for extracting spatial features of a said text-line image;
• a decoding path for extracting textual content from the text-line image;
• said CNN backbone consisting of sequential convolutional layers composed of convolutional, rectified-linear-unit (ReLU), and pooling operations for generating a series of multi-channel feature maps from the text-line image;
• each said convolutional layer containing feature maps having the same spatial resolution and the same number of channels, while its subsequent convolutional layer containing feature maps having an equal or lower spatial resolution and a larger number of channels;
• said decoding path consisting of one or more one-dimensional convolutional layers, a softmax operation, and a Connectionist Temporal Classification (CTC) loss function, providing textual content of the text-line image as output.

A method for automatic detection of text lines in pixelated digital images, comprising:

• a convolutional neural network (CNN) backbone for extracting spatial features of a said image;
• a text-slice detection layer for detecting text-slice candidates;
• a filtering layer for selecting most likely text-slice candidates;
• and
• a text-construction layer for associating text-slice candidates into text lines;
• said CNN backbone consisting of sequential convolutional layers composed of convolutional, rectified-linear-unit (ReLU), and pooling operations for generating a series of multi-channel feature maps from the image;
• each said convolutional layer containing feature maps having the same spatial resolution and the same number of channels, while its subsequent convolutional layer containing feature maps having an equal or lower spatial resolution and a larger number of channels;
• said text-slice detection layer probing one of said feature maps by applying multiple text-slice detectors onto every cell of the probed feature map, with said text-slice detectors having a fixed width equal to the cell width and multiple heights corresponding to aspect ratios in the range of 1 through 8;
• said filtering layer using Non-Maximum Suppression for selecting the best text-slice candidates among those detected by said text-slice detection;
• and
• said text-construction layer grouping adjacent text-slice candidates from said filtering layer having a horizontal distance less than a preset value and a vertical overlap exceeding a preset value into the same text line.

A method for automatic detection of text lines in pixelated digital images, comprising:

• a convolutional neural network (CNN) consisting of a convolutional path followed by a transposed-convolutional path for generating intermediate, pixelated text-distribution maps from a said image;
• and
• a tuning layer for providing an optimal pixelated text-distribution map as output;
• said convolutional path consisting of sequential convolutional layers composed of convolutional, rectified-linear-unit (ReLU), and pooling operations for generating a series of multi-channel feature maps from the image;
• each said convolutional layer containing feature maps having the same spatial resolution and the same number of channels, while its subsequent convolutional layer containing feature maps having an equal or lower spatial resolution and a larger number of channels;
• said transposed-convolutional path consisting of sequential transposed-convolutional layers composed of transposed-convolutional, concatenation, convolutional, and ReLU operations for generating a series of multi-channel feature maps;
• each said transposed-convolutional layer containing feature maps having the same spatial resolution, with its first feature map generated by a transposed-convolutional operation on the last feature map of a lower resolution from its preceding layer, followed by a concatenation operation with a feature map of the same resolution from a said convolutional layer;
• said CNN generating from the image two intermediate, pixelated text-distribution maps, one of original text and the other of eroded text;
• and
• said tuning layer applying Watershed algorithm on the two intermediate text-distribution maps, yielding, as output, an optimal pixelated text-distribution map free of overlapping text.