Back to results
US 8,699,819 B1Granted
Mosaicing documents for translation using video streams
AssigneeGoogle Inc.
InventorsAndrew Moore Ziegler, Leon Palm
Life of this patent
Estimating the term…
Abstract
Aspects of the present disclosure propose techniques for reconstructing a document mosaic using video streams of the document. The streams provide information identifying a layout that relates sequential frames of a video to each other. Once the streams are captured using a mobile device, it is then possible to reconstruct a virtual view of the entire document as though it were taken with a single camera shot. The reconstructed virtual view of the document will be suitable as input to an optical character recognition engine, which can be used for translating the document.
Claims20
A method, comprising:
- receiving a first image of a first view of a document, and a second image of a different, second view of the document;
- selecting multiple visual features from the first image, and multiple visual features from the second image;
- selecting, for each of the multiple visual features from the first image, a set of pixels that are located adjacent to the visual feature from the first image;
- selecting, for each of the multiple visual features from the second image, a set of pixels that are located adjacent to the visual feature from the second image;
- comparing, by one or more computers, each of the sets of pixels from the first image to each of the sets of pixels from the second image to select pairs of sets of pixels that are characterized as matching;
- determining, by the one or more computers, that a quantity of a subset of the pairs of sets of pixels that include (i) a set of pixels that are associated with a first grid location within the first image, and (ii) a set of pixels that are associated with a second grid location within the second image, satisfies a threshold;
- mapping, by the one or more computers, the first grid location within the first image to the second grid location within the second image based on determining that the quantity of a subset of the pairs of sets of pixels that include (i) a set of pixels that are associated with a first grid location within the first image, and (ii) a set of pixels that are associated with a second grid location within the second image, satisfies the threshold;
- and
- generating a composite view of the document using the first image and the second image based on mapping the first grid location within the first image to the second grid location within the second image.
6 more elements
A system comprising:
- one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
- receiving a first image of a first view of a document, and a second image of a different, second view of the document;
- selecting multiple visual features from the first image, and multiple visual features from the second image;
- selecting, for each of the multiple visual features from the first image, a set of pixels that are located adjacent to the visual feature from the first image;
- selecting, for each of the multiple visual features from the second image, a set of pixels that are located adjacent to the visual feature from the second image;
- comparing each of the sets of pixels from the first image to each of the sets of pixels from the second image to select pairs of sets of pixels that are characterized as matching;
- determining that a quantity of a subset of the pairs of sets of pixels that include (i) a set of pixels that are associated with a first grid location within the first image, and (ii) a set of pixels that are associated with a second grid location within the second image, satisfies a threshold;
- mapping the first grid location within the first image to the second grid location within the second image based on determining that the quantity of a subset of the pairs of sets of pixels that include (i) a set of pixels that are associated with a first grid location within the first image, and (ii) a set of pixels that are associated with a second grid location within the second image, satisfies the threshold;
- and
- generating a composite view of the document using the first image and the second image based on mapping the first grid location within the first image to the second grid location within the second image.
7 more elements
A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
- receiving a first image of a first view of a document, and a second image of a different, second view of the document;
- selecting multiple visual features from the first image, and multiple visual features from the second image;
- selecting, for each of the multiple visual features from the first image, a set of pixels that are located adjacent to the visual feature from the first image;
- selecting, for each of the multiple visual features from the second image, a set of pixels that are located adjacent to the visual feature from the second image;
- comparing each of the sets of pixels from the first image to each of the sets of pixels from the second image to select pairs of sets of pixels that are characterized as matching;
- determining that a quantity of a subset of the pairs of sets of pixels that include (i) a set of pixels that are associated with a first grid location within the first image, and (ii) a set of pixels that are associated with a second grid location within the second image, satisfies a threshold;
- mapping the first grid location within the first image to the second grid location within the second image based on determining that the quantity of a subset of the pairs of sets of pixels that include (i) a set of pixels that are associated with a first grid location within the first image, and (ii) a set of pixels that are associated with a second grid location within the second image, satisfies the threshold;
- and
- generating a composite view of the document using the first image and the second image based on mapping the first grid location within the first image to the second grid location within the second image.
6 more elements
Dependencies are inferred from the claim text.