# US 10,769,431 B2 · Handheld device for capturing text from both a document printed on paper and a document displayed on a dynamic display device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,431 B2 - Document ID: US10769431B2 - Status: Granted - Kind: B2 - Application number: 16/133,654 - Filed: 2018-09-17 - Published: 2019-03-21 - Granted: 2020-09-08 - Earliest claimed: 2004-09-27 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Martin T. King; Dale L. Grover; Clifford A. Kushler; James Q. Stafford-Fraser - Web page: https://patentgrep.com/patent/US10769431B2 ## CPC - H04N 1/32101: {Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title} - G06F 16/5846: {using extracted text} - G06Q 10/10: Office automation; Time management - G06Q 30/00: Commerce - G06V 10/95: {structured as a network, e.g. client-server architectures} - G06V 20/66: Trinkets, e.g. shirt buttons or jewellery items (recognising microscopic objects G06V20/69) - G06V 30/142: using hand-held instruments; Constructional details of the instruments - G06V 30/36: {Matching; Classification} - G06V 30/416: Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors - G06V 30/418: Document matching, e.g. of document images - H04N 1/107: with manual scanning - H04N 1/00241: {using an image reading device as a local input to a computer (input via network, e.g. network scanners H04N1/00225)} - H04N 1/00307: {with a mobile telephone apparatus (constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00)} - H04N 1/00488: {providing an audible output to the user} - H04N 1/0049: {providing a visual indication to the user, e.g. using a lamp (H04N1/00408 takes precedence)} - H04N 2201/0041: Point to point (H04N2201/0039 takes precedence) - H04N 2201/0055: By radio - H04N 2201/0058: Docking-station, cradle or the like - H04N 2201/0081: Image reader (H04N2201/0091 - H04N2201/0094 take precedence) - H04N 2201/0084: Digital still camera - H04N 2201/0096: Portable devices - H04N 2201/3249: data relating to a linked page or object, e.g. hyperlink - H04N 2201/3264: of sound signals - H04N 2201/3269: of machine readable codes or marks, e.g. bar codes or glyphs - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A device for capturing rendered text is described. The device incorporates one or more visual sensors that receive visual information as a part of capturing rendered text. The visual sensors are collectively capable of capturing both text that is permanently printed on a page, and text that is displayed transitorily on a dynamic device. The device further incorporates a visual information disposition subsystem for disposing of visual information received by the visual sensors. The device further incorporates a package that bears the visual sensors and the visual information disposition subsystem, and is suitable to be held in a human hand. ## Claims ### Claim 1 A computer-implemented method to capture data from digital images, comprising: receiving, by one or more computing devices, an input to capture digital images; providing, by the one or more computing devices, a user interface that displays instructions to capture digital images; initiating, by the one or more computing devices, an image capturing module of the one or more computing devices; receiving, by the one or more computing devices, a digital image from the image capturing module; determining, by the one or more computing devices, that at least a portion of a set of text in the digital image is readable; performing, by the one or more computing devices, an optical character recognition algorithm on text in the digital image; and providing, by the one or more computing devices, a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface. ### Claim 2 (depends on 1) The computer implemented method of claim 1, further comprising: receiving, by the one or more computing devices, an input of an annotation to the digital image; and communicating, by the one or more computing devices, the annotation with the digital image to a processing system. ### Claim 3 (depends on 1) The computer implemented method of claim 1, further comprising: determining, by the one or more computing devices, that at least a portion of a set of text in the digital image is not viewable; and providing, by the one or more computing devices, a notification on the user interface to obtain a second digital image. ### Claim 4 (depends on 1) The computer implemented method of claim 1, further comprising: providing, by the one or more computing devices, results of the optical character recognition algorithm to a processing system. ### Claim 5 (depends on 1) The computer implemented method of claim 1, wherein the instructions are provided in a pop-up menu. ### Claim 6 (depends on 1) The computer implemented method of claim 1, further comprising communicating, by the one or more computing devices, a request to a processing system to initiate a financial transaction in connection with the digital image. ### Claim 7 (depends on 1) The computer implemented method of claim 1, wherein the one or more computing devices comprise a mobile computing device. ### Claim 8 (depends on 7) The computer implemented method of claim 7, wherein the image capturing module is a function of the mobile computing device. ### Claim 9 A computer program product, comprising: a non-transitory computer-readable storage device having computer-executable program instructions embodied thereon that when executed by one or more computing devices cause the computer to capture data from digital images, the computer-readable program instructions comprising computer-executable instructions to: receive an input to capture digital images; provide a user interface that displays instructions to capture digital images; initiate an image capturing module of the one or more computing devices; receive a digital image from the image capturing module; determine that at least a portion of a set of text in the digital image is readable; perform an optical character recognition algorithm on text in the digital image; and provide a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface. ### Claim 10 (depends on 9) The computer program product of claim 9, further comprising computer-executable instructions to: receive an input of an annotation to the digital image; and communicate the annotation with the digital image to a processing system. ### Claim 11 (depends on 9) The computer program product of claim 9, further comprising computer-executable instructions to: determine that at least a portion of a set of text in the digital image is not viewable; and provide a notification on the user interface to obtain a second digital image. ### Claim 12 (depends on 10) The computer program product of claim 10, further comprising computer-executable instructions to: provide the results of the optical character recognition algorithm to a processing system. ### Claim 13 A system to capture data from digital images, comprising: a storage device; and a processor communicatively coupled to the storage device, wherein the processor executes application code instructions that are stored in the storage device to cause the system to: receive an input to capture digital images; provide a user interface that displays instructions to capture digital images; initiate an image capturing module of the processor; receive a digital image from the image capturing module; determine that at least a portion of a set of text in the digital image is readable; perform an optical character recognition algorithm on text in the digital image; and provide a display of at least a portion of the results of the optical character recognition algorithm as an overlay to the digital image on the user interface. ### Claim 14 (depends on 13) The system of claim 13, wherein the instructions are provided in a pop-up menu. ### Claim 15 (depends on 13) The system of claim 13, wherein the processor executes application code instructions that are stored in the storage device to cause the system to communicate a request to a processing system to initiate a financial transaction in connection with the digital image. ### Claim 16 (depends on 13) The system of claim 13, wherein the system comprises a mobile computing device. ### Claim 17 (depends on 13) The system of claim 13, wherein the processor executes application code instructions that are stored in the storage device to cause the system to communicate the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable. ### Claim 18 (depends on 1) The computer implemented method of claim 1, further comprising: communicating, by the one or more computing devices, the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable. ### Claim 19 (depends on 9) The computer program product of claim 9, further comprising computer-executable instructions to communicate the digital image to a processing system in response to determining that at least a portion of the set of text in the digital image is readable.