# US 12,112,140 B2 · Electronic device for outputting translated text with improved accuracy and control method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,140 B2
- Document ID: US12112140B2
- Status: Granted
- Kind: B2
- Application number: 17/666,127
- Filed: 2022-02-07
- Published: 2022-08-04
- Granted: 2024-10-08
- Earliest claimed: 2021-02-04
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Beomseok Lee; Hyojung Han; Sathish Indurthi; Mohd Abbas Zaidi; Nikhil Kumar
- Web page: https://patentgrep.com/patent/US12112140B2

## CPC

- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/30: Semantic analysis
- G06F 40/44: Statistical methods, e.g. probability models
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G10L 13/02: Methods for producing synthetic speech; Speech synthesisers
- G10L 13/027: Concept to speech synthesisers; Generation of natural phrases from machine-based concepts (generation of parameters for speech synthesis out of text G10L13/08)
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An electronic device includes a memory storing at least one instruction, and a processor configured to execute the at least one instruction to: input a first text into a first neural network model and acquire a first translation text; input the first translation text into a second neural network model and acquire a first liberal translation text; identify whether the first liberal translation text is to be outputted based on the first text and the first liberal translation text; output the first liberal translation text according to the identification result; based on a second text being received subsequently to the first text, input the first and second texts into the first neural network model and acquire a second translation text; and input the identification result and the second translation text into the second neural network model and acquire a second liberal translation text corresponding to the second translation text.

## Claims

### Claim 1

An electronic device comprising:
a speaker;
a memory configured to store at least one instruction; and
a processor electronically configured to execute the at least one instruction to:
input a first text in a first language into a first neural network model and acquire a first translation text corresponding to the first text, the first translation text being in a second language that is different from the first language;
input the first translation text into a second neural network model and acquire a first liberal translation text corresponding to the first translation text, the first liberal translation text being a text obtained by including at least one that is different from words included in the first translation text and/or rearranging words included in the first translation text;
identify whether the first liberal translation text is to be output based on a sentence similarity value between the first translation text and the first liberal translation text, and further based on a sentence suitability value of the first liberal translation text, to obtain a first identification result;
selectively control the speaker to output a sound corresponding to the first liberal translation text based on whether the first identification result indicates that the first liberal translation text is to be output;
based on a second text being received subsequently to the first text, input the first text and the second text into the first neural network model and acquire a second translation text corresponding to the first text and the second text; and
input the first identification result and the second translation text into the second neural network model and acquire a second liberal translation text corresponding to the second translation text, the second liberal translation text being a text obtained by including at least one that is different from words included in the second translation text and/or rearranging words included in the second translation text, and
wherein the second neural network model is configured to:
based on identifying that the sound corresponding to the first liberal translation text was previously output via the speaker, according to the first identification result, output the second liberal translation text corresponding to the second translation text, the second liberal translation text including the first liberal translation text, which is located in front in a word order in the second liberal translation text; and
based on identifying that the sound corresponding to the first liberal translation text was not previously output via the speaker, according to the first identification result, output the second liberal translation text corresponding to the second translation text, the second liberal translation text not including the first liberal translation text in front in a word order in the second liberal translation text.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to execute the at least one instruction to:
based on the sentence similarity value being greater than or equal to a first threshold value, and further based on the sentence suitability value being greater than or equal to a second threshold value, control the speaker to output the sound corresponding to the first liberal translation text.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to execute the at least one instruction to:
identify whether the second liberal translation text is to be output based on the second text and the second liberal translation text to obtain a second identification result,
based on identifying that the sound corresponding to the first liberal translation text was previously output via the speaker, and the second liberal translation text is to be output according to the second identification result, control the speaker to output a sound corresponding to a remaining part excluding the first liberal translation text in the second liberal translation text, and
based on identifying that the sound corresponding to the first liberal translation text was not previously output via the speaker, and the second liberal translation text is to be output, control the speaker to output a sound corresponding to the second liberal translation text.

### Claim 4 (depends on 3)

The electronic device of claim 3, wherein the processor is further configured to execute the at least one instruction to:
acquire a sentence similarity value between the second translation text and the second liberal translation text,
acquire a sentence suitability value of the second liberal translation text,
identify whether the second liberal translation text is to be output based on the sentence similarity value between the second translation text and the second liberal translation text and the sentence suitability value, and
based on the sentence similarity value between the second translation text and the second liberal translation text being greater than or equal to a third threshold value and the sentence suitability value being greater than or equal to a fourth threshold value, control the speaker to output the sound corresponding to the second liberal translation text.

### Claim 5 (depends on 1)

The electronic device of claim 1, wherein the processor is further configured to execute the at least one instruction to:
input the first text and the second text into a third neural network model and acquire information on whether the first text and the second text correspond to a phrase, and
input the second translation text and the information on whether the first text and the second text correspond to the phrase into the second neural network model and acquire the second liberal translation text corresponding to the second translation text.

### Claim 6 (depends on 5)

The electronic device of claim 5, wherein the second neural network model is further configured to:
based on identifying that the first text and the second text correspond to the phrase based on the input information, output the second liberal translation text corresponding to the second translation text and in a form of a sentence or the phrase, and
based on identifying that the first text and the second text do not correspond to the phrase, output the second liberal translation text corresponding to the second translation text and in a form of an uncompleted sentence.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein the second neural network model is trained based on a plurality of texts,
wherein the plurality of texts comprise:
a first sentence text acquired based on at least one word selected from a plurality of respective word groups and a second sentence text of which sentence similarity value with respect to the first sentence text is greater than or equal to a fifth threshold value, and
wherein the plurality of respective word groups include words of which sentence similarity values therebetween are greater than or equal to the fifth threshold value.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein the second neural network model is trained based on a plurality of texts,
wherein the plurality of texts comprise:
a first translation sentence text in the second language corresponding to a first sentence text in the first language acquired based on a parallel corpus between the first language and the second language; and
a second translation sentence text in the second language acquired by inputting the first sentence text into the first neural network model, and
wherein a sentence similarity value between the first translation sentence text and the second translation sentence text is greater than or equal to a sixth threshold value.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein a most preceding word among a plurality of words constituting the first translation sentence text and a most preceding word among a plurality of words constituting the second translation sentence text are different words from each other.

### Claim 10

A control method of an electronic device, the control method comprising:
inputting a first text in a first language into a first neural network model and acquiring a first translation text corresponding to the first text, the first translation text being in a second language that is different from the first language;
inputting the first translation text into a second neural network model and acquiring a first liberal translation text corresponding to the first translation text, the first liberal translation text being a text obtained by including at least one that is different from words included in the first translation text and/or rearranging words included in the first translation text;
identifying whether the first liberal translation text is to be output via a speaker based on a sentence similarity value between the first translation text and the first liberal translation text, and further based on a sentence suitability value of the first liberal translation text, to obtain a first identification result;
selectively controlling the speaker to output a sound corresponding to the first liberal translation text based on whether the first identification result indicates that the first liberal translation text is to be output;
based on a second text being received subsequently to the first text, inputting the first text and the second text into the first neural network model and acquiring a second translation text corresponding to the first text and the second text; and
inputting the first identification result and the second translation text into the second neural network model and acquiring a second liberal translation text corresponding to the second translation text, the second liberal translation text being a text obtained by including at least one that is different from words included in the second translation text and/or rearranging words included in the second translation text,
wherein the second neural network model is configured to:
based on identifying that the sound corresponding to the first liberal translation text was previously output via the speaker according to the first identification result, output the second liberal translation text corresponding to the second translation text, the second liberal translation text including the first liberal translation text, which is located in front in a word order in the second liberal translation text; and
based on identifying that the sound corresponding to the first liberal translation text was not previously output via the speaker, output the second liberal translation text corresponding to the second translation text, the second liberal translation text not including the first liberal translation text in front in a word order in the second liberal translation text.

### Claim 11 (depends on 10)

The control method of claim 10,
wherein the outputting the first liberal translation text comprises:
based on the sentence similarity value being greater than or equal to a first threshold value, and further based on the sentence suitability value being greater than or equal to a second threshold value, outputting the first liberal translation text.

### Claim 12 (depends on 10)

The control method of claim 10, further comprising:
identifying whether the second liberal translation text is to be output based on the second text and the second liberal translation text to obtain a second identification result;
based on identifying that the sound corresponding to the first liberal translation text was previously output via the speaker, and the second liberal translation text is to be output according to the second identification result, outputting a remaining part excluding the first liberal translation text in the second liberal translation text; and
based on identifying that the sound corresponding to the first liberal translation text was not previously output via the speaker, and the second liberal translation text is to be output, outputting the second liberal translation text.

### Claim 13 (depends on 12)

The control method of claim 12,
wherein the identifying whether the second liberal translation text is to be output comprises:
acquiring a sentence similarity value between the second translation text and the second liberal translation text;
acquiring a sentence suitability value of the second liberal translation text; and
identifying whether the second liberal translation text is to be output based on the sentence similarity value between the second translation text and the second liberal translation text and the sentence suitability value, and
wherein the outputting the second liberal translation text comprises:
based on the sentence similarity value between the second translation text and the second liberal translation text being greater than or equal to a third threshold value and the sentence suitability value being greater than or equal to a fourth threshold value, outputting the second liberal translation text.

### Claim 14 (depends on 10)

The control method of claim 10, further comprising:
inputting the first text and the second text into a third neural network model and acquire information on whether the first text and the second text correspond to a phrase, and
wherein the identifying and acquiring the second liberal translation text corresponding to the second translation text comprises:
inputting the second translation text and the information on whether the first text and the second text correspond to the phrase into the second neural network model and acquiring the second liberal translation text corresponding to the second translation text.

### Claim 15 (depends on 14)

The control method of claim 14, wherein the second neural network model is further configured to:
based on identifying that the first text and the second text correspond to the phrase based on the input information, output the second liberal translation text corresponding to the second translation text and in a form of a sentence or the phrase, and
based on identifying that the first text and the second text do not correspond to the phrase, output the second liberal translation text corresponding to the second translation text and in a form of an uncompleted sentence.

### Claim 16 (depends on 10)

The control method of claim 10, wherein the second neural network model is trained based on a plurality of texts,
wherein the plurality of texts comprise:
a first sentence text acquired based on at least one word selected from a plurality of respective word groups and a second sentence text of which sentence similarity value with respect to the first sentence text is greater than or equal to a fifth threshold value, and
wherein the plurality of respective word groups include words of which sentence similarity values therebetween are greater than or equal to the fifth threshold value.

### Claim 17 (depends on 10)

The control method of claim 10, wherein the second neural network model is trained based on a plurality of texts,
wherein the plurality of texts comprise:
a first translation sentence text in the second language corresponding to a first sentence text in the first language acquired based on a parallel corpus between the first language and the second language; and
a second translation sentence text in the second language acquired by inputting the first sentence text into the first neural network model, and
wherein a sentence similarity value between the first translation sentence text and the second translation sentence text is greater than or equal to a sixth threshold value.

### Claim 18 (depends on 17)

The control method of claim 17, wherein a most preceding word among a plurality of words constituting the first translation sentence text and a most preceding word among a plurality of words constituting the second translation sentence text are different words from each other.