# US 2018/0322677 A1 · Information processing device, storage medium, and method of operating information processing device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322677 A1
- Document ID: US20180322677A1
- Status: Published application
- Kind: A1
- Application number: 16/037,280
- Filed: 2018-07-17
- Published: 2018-11-08
- Earliest claimed: 2013-09-27
- Assignees: Sharp Kabushiki Kaisha
- Applicants: Sharp Kabushiki Kaisha
- Inventors: Kiyofumi OHTSUKA; Tadao NAGASAWA
- Web page: https://patentgrep.com/patent/US20180322677A1

## CPC

- G06F 40/109: Font handling; Temporal or kinetic typography
- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06T 11/60: Creating or editing images; Combining images with text
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present invention makes it possible to see an original text string which (i) is contained in a captured image and (ii) has been translated, even after a translation of the original text string is displayed. The text string decoration display control section (12) decorates a part indicating a text string contained in a captured image and causes the decorated part to be displayed. The translation image generating section (13) generates a translation image showing a result of translating the text string into another language. The translation display control section (16) switches between display and non-display of the translation image in accordance with an input carried out by a user.

## Claims

### Claim 1

An information processing device comprising:
an image capturing device that optically captures an image;
a display device that displays, on a screen, the image captured by the image capturing device;
an operation button displayed in the display device; and
at least one processor,
the at least one processor being configured to, after the operation button have received an operation, decorate a text string image word by word to obtain a decorated text string image and control the display device to display the decorated text string image, the text string image being an image of a text string which has been successfully extracted from the image captured by the image capturing device.

### Claim 2

An information processing device comprising:
an image capturing device that optically captures an image;
a display device that displays, on a screen, the image captured by the image capturing device;
an operation button displayed in the display device; and
at least one processor,
the at least one processor being configured to, after the operation button have received an operation, if the at least one processor has failed to extract a text string contained in the image captured by the image capturing device, notify a user that the at least one processor has failed to extract the text string.

### Claim 3

An information processing device comprising:
an image capturing device that optically captures an image;
a display device that displays, on a screen, the image captured by the image capturing device;
an operation button displayed in the display device; and
at least one processor,
the at least one processor being configured to, before the operation button receives an operation, control the display device to display a text string image and a translation image on the screen such that the translation image is superimposed on the text string image, the text string image being an image of a text string contained in the image captured by the image capturing device, the translation image being an image showing a result of translation of the text string.

### Claim 4 (depends on 1)

A non-transitory computer-readable storage medium storing a control program for causing a computer to function as an information processing device recited in claim 1, the control program causing the computer to function as the at least one processor.

### Claim 5 (depends on 2)

A non-transitory computer-readable storage medium storing a control program for causing a computer to function as an information processing device recited in claim 2, the control program causing the computer to function as the at least one processor.

### Claim 6 (depends on 3)

A non-transitory computer-readable storage medium storing a control program for causing a computer to function as an information processing device recited in claim 3, the control program causing the computer to function as the at least one processor.

### Claim 7

A method of operating an information processing device, the method comprising the steps of:
optically capturing an image;
displaying the image on a screen;
displaying an operation button on the screen; and
after the operation button have received an operation, decorating a text string image word by word to obtain a decorated text string image and displaying the decorated text string image on the screen, the text string image being an image of a text string which has been successfully extracted from the image.

### Claim 8

A method of operating an information processing device, the method comprising the steps of:
optically capturing an image;
displaying the image on a screen;
displaying an operation button on the screen; and
after the operation button have received an operation, carrying out an extraction of a text string contained in the image and, if the extraction of the text string has failed, notifying a user that the extraction of the text string has failed.

### Claim 9

A method of operating an information processing device, the method comprising the steps of:
optically capturing an image;
displaying the image on a screen;
displaying an operation button on the screen; and
before the operation button receives an operation, controlling the display device to display a text string image and a translation image on the screen such that the translation image is superimposed on the text string image, the text string image being an image of a text string contained in the image, the translation image being an image showing a result of translation of the text string.
