US 9,292,739 B1Granted

Automated recognition of text utilizing multiple images

AssigneeA9.com, Inc.

InventorsDouglas Ryan Gray, Colin Jay Taylor, Xiaofan Lin, Adam Wiggen Kraft, Yu Lou, Arnab Sanat Kumar Dhua

Life of this patent

Estimating the term…

Loading patent…

Abstract

Various embodiments enable text aggregation from multiple image frames of text. Accordingly, in order to stitch newly scanned areas of a document together, text in a respective image is recognized and analyzed using an algorithm to identify pairs of corresponding words in other images. Upon identifying a minimum number of matching pairs between two respective images, a mapping between the same can be determined based at least in part on a geometric correspondence between respective identified pairs. Based on this mapping, the recognized text of the two images can be merged by adding words of one image to the other using the matching word pairs as alignment data points.

Claims18

Dependent claims

A computing device, comprising:

a processor;
a camera;
a display screen;
and
memory including instructions that, when executed by the processor, cause the computing device to performing a set of operations comprising:
capture, using the camera, a first image of a first portion of a body of text, at least a portion of the body of text is outside a field of view of the camera;
recognize, using an optical character recognition (OCR) engine with the first image, text of the first portion of the body of text;
generate a text canvas using the recognized text of the first portion, the text canvas comprising a plurality of text strings;
capture, using the camera, a second image of a second portion of the body of text, the second portion partially overlapping the first portion;
recognize, using the OCR engine with the second image, text of the second portion of the body of text;
identify pairs of corresponding words in the recognized text of the first portion and the recognized text of the second portion;
determine a mapping between the recognized text of the first portion and the recognized text of the second portion based at least in part on a geometric correspondence between respective identified pairs of corresponding words;
merge, based at least in part on the determined mapping, at least a subset of the second portion of the body of text with the text canvas by merging one or more words of the recognized text of the second portion to the text canvas and, as subsequent images are obtained, the text canvas is iteratively updated as new images of the body of text are captured based at least in part on confidence scores;
and
display, on the display screen, the text canvas.

12 more elements

A computer-implemented method, comprising:

obtaining at least one image of a body of text;
recognizing text included in the at least one image;
comparing the recognized text to a set of representations of text corresponding to one or more bodies of text;
identifying a representation of text from among the set of representations of text having at least a portion of text in common with the recognized text of the at least one image, the representation of text composed of a plurality of text strings;
determining a mapping between the recognized text of the at least one image and the representation of text based at least on a geometric correspondence between identified pairs of corresponding text strings in the recognized text of the at least one image and the representation of text;
and
merging, based at least on the determined mapping, one or more text strings of the recognized text of the at least one image to the representation of text and, as subsequent images are obtained, iteratively updating the representation of text as new images of the body of text are captured based at least in part on confidence scores.

4 more elements

A computer-implemented method, comprising:

obtaining a first image of a first portion of a body of text;
recognizing text of the first portion of the body of text in the first image;
obtaining a second image of a second portion of the body of text, the second portion of the body of text including an overlapping portion where the second portion of the body of text partially overlaps the first portion;
recognizing text of the second portion of the body of text in the second image;
identifying, in the overlapping portion, corresponding words in the recognized text of the first portion and the recognized text of the second portion;
and
merging, based at least in part on the corresponding words in the overlapping portion, the recognized text of the second portion with the recognized text of the first portion to generate a representation of the body of text, the representation of the body of text including a plurality of words and, as subsequent images are obtained, the representation of the body of text is iteratively updated as new images of the body of text are obtained based at least in part on confidence scores.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/105,084Filed 12 Dec 2013
Granted: 22 Mar 2016
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G06V 30/142
Image acquisition: using hand-held instruments; Constructional details of the instruments
G06F 18/254
Fusion techniques: of classification results, e.g. of results related to same input data
G06V 30/127
Detection or correction of errors, e.g. by rescanning the pattern: with the intervention of an operator
G06V 30/1918
Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation:…