# US 2014/0372100 A1 · Translation system comprising display apparatus and server and display apparatus controlling method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0372100 A1
- Document ID: US20140372100A1
- Status: Published application
- Kind: A1
- Application number: 14/308,141
- Filed: 2014-06-18
- Published: 2014-12-18
- Earliest claimed: 2013-06-18
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Jae-yun JEONG; Sung-jin KIM; Yong-gyoo KIM
- Web page: https://patentgrep.com/patent/US20140372100A1

## CPC

- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G10L 21/10: Transforming into visible information
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 2021/105: {Synthesis of the lips movements from speech, e.g. for talking heads}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A display apparatus is provided. The display apparatus includes: an inputter configured to receive a user's face shape and voice; a voice processor configured to analyze the input voice and extract translated data, and convert the translated data into translated voice; an image processor configured to detect information related to a mouth area of the user's face shape which corresponds to the translated data, and create a changed shape of a user's face based on the detected information related to the mouth area; and an outputter configured to output the translated voice and the changed shape of the user's face.

## Claims

### Claim 1

A display apparatus comprising:
an inputter configured to receive a user's face shape and voice;
a voice processor configured to analyze the input voice and extract translated data, and convert the translated data into translated voice;
an image processor configured to detect information related to a mouth area of the user's face shape which corresponds to the translated data, and create a changed shape of a user's face based on the detected information related to the mouth area; and
an outputter configured to output the translated voice and changed shape of the user's face.

### Claim 2 (depends on 1)

The display apparatus according to claim 1,
wherein the image processor is configured to synchronize the changed shape of the user's face to the translated voice.

### Claim 3 (depends on 1)

The display apparatus according to claim 1,
wherein the voice processor is configured to compare a length of the input voice and a length of the translated voice and adjust the length of the translated voice based on the comparison.

### Claim 4 (depends on 1)

The display apparatus according to claim 1,
wherein the voice processor is configured to extract at least one characteristic of a tone, pitch and sound quality of the input voice and applies the extracted characteristic to the translated voice.

### Claim 5 (depends on 1)

The display apparatus according to claim 1,
wherein the information related to the mouth area is configured to be mesh information where characteristic points of the stored mouth-shaped image are connected, and
the image processor extracts a phoneme from the translated data and searches for a corresponding mouth-shaped image, and maps the mesh information where the characteristic points of the searched mouth-shaped image are connected to the user's face shape in order to create a changed shape of the user's face.

### Claim 6 (depends on 1)

The display apparatus according to claim 1,
wherein the information related to the mouth area is a stored mouth-shaped image, and
the image processor is configured to extract a phoneme from the translated data and searches for a corresponding mouth-shaped image, and edits the searched mouth-shaped image in the face shape to create a changed shape of the user's face.

### Claim 7 (depends on 1)

The display apparatus according to claim 1,
further comprising:
a communicator configured to communicate with a server,
wherein the communicator transmits the user's face shape and input voice to the server, and receives from the server the translated voice and changed shaped of the user's face.

### Claim 8 (depends on 1)

The display apparatus according to claim 1,
further comprising a communicator configured to communicate with a server,
wherein the communicator is configured to transmit the user's face shape and input voice to the server, and receives the translated voice and mouth area information from the server, and
the image processor is configured to create the changed shape of the user's face based on the received information related to the mouth area.

### Claim 9

A server configured to communicate with a display apparatus, the server comprising:
a communicator configured to receive from the display apparatus a user's face shape and voice;
a voice processor configured to analyze the received voice and extract translated data, and convert the translated data into translated voice; and
an image processor configured to detect information related to a mouth area of the user's face shape which corresponds to the translated data,
wherein the communicator is configured to transmit the information related to the mouth area to the display apparatus together with the translated voice.

### Claim 10 (depends on 9)

The server according to claim 9,
wherein the image processor is configured to create a changed shape of a user's shape based on the detected information related to the mouth area, and
the communicator is configured to transmit the changed shape of the user's shape to the display apparatus together with the translated voice.

### Claim 11

A conversion system comprising a display apparatus and a server, the system comprising:
a display apparatus configured to transmit an input shape of a user's face and input voice to the server; and
a server configured to analyze the input voice and extract translated data and convert the translated data into translated voice, and detect information related to a mouth area of the user's face shape which corresponds to the translated data in order to create a changed shape of the user's face mapped from the user's face shape.
wherein the display apparatus is configured to receive from the server the changed shape of the user's face or information related to the mouth area, together with the translated voice.

### Claim 12

A method of controlling a display apparatus, the method comprising:
receiving a user's face shape and voice;
analyzing the input voice and extracting the translated data;
detecting information related to a mouth area of the user's face shape which corresponds to the translated data, and creating a changed shape of a user's face based on the detected information related to the mouth area;
converting the translated data into translated voice; and
outputting the translated voice and the changed shape of the user's face.

### Claim 13 (depends on 12)

The method according to claim 12,
wherein the outputting synchronizes the changed shape of the user's face with the translated voice.

### Claim 14 (depends on 12)

The method according to claim 12,
further comprising comparing a length of the input voice and a length of the translated voice and adjusting the length of the translated voice based on the comparison.

### Claim 15 (depends on 12)

The method according to claim 12,
further comprising extracting at least one characteristic of a tone, pitch and sound quality of the input voice and applying the extracted characteristic to the translated voice.

### Claim 16 (depends on 12)

The method according to claim 12,
wherein the information related to the mouth area is mesh information where characteristic points of the stored mouth-shaped image are connected, and
the creating a changed shape of a user's face extracts a phoneme from the translated data and searches for a corresponding mouth-shaped image, and maps the mesh information where the characteristic points of the searched mouth-shaped image are connected to the user's face shape in order to create a changed shape of a user's face.

### Claim 17 (depends on 12)

The method according to claim 12,
wherein the information related to the mouth area is a stored mouth-shaped image, and
the creating a changed shape of the user's face extracts a phoneme from the translated data and searches for a corresponding mouth-shaped image, and edits the searched mouth-shaped image in the face shape in order to create a changed shape of a user's face.

### Claim 18 (depends on 12)

The method according to claim 12,
further comprising transmitting to the server the user's face shape and input voice, and receiving from the server the translated voice and changed shape of the user's face shape.
wherein the outputting outputs the received translated voice and received changed shape of the user's face.

### Claim 19 (depends on 12)

The method according to claim 12,
further comprising transmitting to the server the user's face shape and input voice, and receiving from the server the translated voice and changed shape of the user's face.
wherein the creating the changed shape of the user's face creates the changed shape of the user's face based on the received information related to the mouth area.

### Claim 20

A display apparatus comprising:
a voice processor configured to analyze an input voice and extract translated data, and convert the translated data into translated voice;
an image processor configured to detect information related to a mouth area of the user's face shape which corresponds to the translated data, and create a changed shape of a user's face based on the detected information.

### Claim 21 (depends on 20)

The display apparatus of claim 20, further comprising an inputter configured to receive a user's face shape and voice.

### Claim 22 (depends on 20)

The display apparatus of claim 20, further comprising an outputter configured to output the translated voice and changed shape of the user's face.

### Claim 23 (depends on 20)

The display apparatus according to claim 20, wherein the image processor is configured to synchronize the changed shape of the user's face to the translated voice.

### Claim 24 (depends on 20)

The display apparatus according to claim 20, wherein the voice processor is configured to compare a length of the input voice and a length of the translated voice and adjust the length of the translated voice based on the comparison.

### Claim 25 (depends on 20)

The display apparatus according to claim 20, wherein the voice processor is configured to extract at least one characteristic of a tone, pitch and sound quality of the input voice and applies the extracted characteristic to the translated voice.

### Claim 26 (depends on 20)

The display apparatus according to claim 20, wherein the information related to the mouth area is configured to be a stored mouth-shaped image.

### Claim 27 (depends on 20)

The display apparatus according to claim 20, wherein the image processor is configured to extract a phoneme from the translated data and searches for a corresponding mouth-shaped image, and edits the searched mouth-shaped image in the face shape to create a changed shape of the user's face.

### Claim 28 (depends on 20)

The display apparatus according to claim 20, further comprising:
a communicator configured to communicate with a server,
wherein the communicator transmits the user's face shape and input voice to the server, and receives from the server the translated voice and changed shaped of the user's face.
