# US 9,436,682 B2 · Techniques for machine language translation of text from an image based on non-textual context information from the image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,682 B2 - Document ID: US9436682B2 - Status: Granted - Kind: B2 - Application number: 14/313,670 - Filed: 2014-06-24 - Published: 2015-12-24 - Granted: 2016-09-06 - Earliest claimed: 2014-06-24 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: Arne Mauser; Alexander Jay Cuthbert; John Sturdy DeNero - Web page: https://patentgrep.com/patent/US9436682B2 ## CPC - G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation - G06V 20/20: in augmented reality scenes - G06V 20/63: {Scene text, e.g. street names} - G06V 30/262: using context analysis, e.g. lexical, syntactic or semantic context - G06V 30/10: Character recognition - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer-implemented technique can include receiving, at a server from a mobile computing device, the server having one or more processors, an image including a text. The technique can include obtaining, at the server, optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image. The technique can include identifying, at the server, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself and (ii) being indicative of a context of the image. The technique can include based on the non-textual context information, obtaining, at the server, a translation of the OCR text to a target language to obtain a translated OCR text. The technique can include outputting, from the server to the mobile computing device, the translated OCR text. ## Claims ### Claim 1 A computer-implemented method, comprising: receiving, at a server from a mobile computing device, the server having one or more processors, an image including a text; obtaining, at the server, optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image; identifying, at the server, non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a color of an object in the image; based on the color of the object, determining, at the server, whether the image was captured indoors or outdoors; based on (i) the non-textual context information and (ii) whether the image was captured indoors or outdoors, obtaining, at the server, a translation of the OCR text to a target language to obtain a translated OCR text; and outputting, from the server to the mobile computing device, the translated OCR text. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: obtaining, at the server, a translation of the OCR text to the target language to obtain a baseline translated OCR text; and adjusting, at the server, the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising determining, at the server, a source language of the text based on the non-textual context information, wherein the translated OCR text is further based on the source language. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising determining, at the server, a type of location at which the image was captured based on the non-textual context information, wherein the translated OCR text is further based on the type of location. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising determining, at the server, a geo-location of the mobile computing device, wherein the translated OCR text is further based on the geo-location of the mobile computing device. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, further comprising: obtaining, at the server, map information based on the geo-location; and identifying, at the server, points of interest near the geo-location using the map information, wherein the translated OCR text is further based on the points of interest near the geo-location. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising determining, at the server, a user history corresponding to a user of the mobile computing device, wherein the translated OCR text is further based on the user history. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the non-textual context information includes a font of the text. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the non-textual context information includes a shape of the object. ### Claim 10 A server having one or more processors configured to perform operations comprising: receiving, from a mobile computing device, an image including a text; obtaining optical character recognition (OCR) text corresponding to the text, the OCR text having been obtained by performing OCR on the image; identifying non-textual context information from the image, the non-textual context information (i) representing context information other than the text itself, (ii) being indicative of a context of the image, and (iii) including at least a color of an object in the image; based on the color of the object, determining whether the image was captured indoors or outdoors; based on (i) the non-textual context information and (ii) whether the image was captured indoors or outdoors, obtaining a translation of the OCR text to a target language to obtain a translated OCR text; and outputting, to the mobile computing device, the translated OCR text. ### Claim 11 (depends on 10) The server of claim 10, wherein the operations further comprise: obtaining a translation of the OCR text to the target language to obtain a baseline translated OCR text; and adjusting the baseline translated OCR text based on the non-textual context information to obtain the translated OCR text. ### Claim 12 (depends on 10) The server of claim 10, wherein the operations further comprise determining a source language of the text based on the non-textual context information, wherein the translated OCR text is further based on the source language. ### Claim 13 (depends on 10) The server of claim 10, wherein the operations further comprise determining a type of location at which the image was captured based on the non-textual context information, wherein the translated OCR text is further based on the type of location. ### Claim 14 (depends on 10) The server of claim 10, wherein the operations further comprise determining a geo-location of the mobile computing device, wherein the translated OCR text is further based on the geo-location of the mobile computing device. ### Claim 15 (depends on 14) The server of claim 14, wherein the operations further comprise: obtaining map information based on the geo-location; and identifying points of interest near the geo-location using the map information, wherein the translated OCR text is further based on the points of interest near the geo-location. ### Claim 16 (depends on 10) The server of claim 10, wherein the operations further comprise determining a user history corresponding to a user of the mobile computing device, wherein the translated OCR text is further based on the user history. ### Claim 17 (depends on 10) The server of claim 10, wherein the non-textual context information includes at least one of (i) a font of the text and (ii) a shape of the object.