# US 12,001,808 B2 · Method and apparatus for providing interpretation situation information to one or more devices based on an accumulated delay among three devices in three different languages

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,808 B2
- Document ID: US12001808B2
- Status: Granted
- Kind: B2
- Application number: 17/435,995
- Filed: 2021-08-23
- Published: 2023-06-15
- Granted: 2024-06-04
- Earliest claimed: 2020-09-02
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Yoonjung Choi; Sangha Kim; Hakjung Kim; Yoonjin Yoon; Seokchan Ahn
- Web page: https://patentgrep.com/patent/US12001808B2

## CPC

- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06N 3/09: Supervised learning
- G06Q 50/10: Services
- G10L 21/04: Time compression or expansion
- G10L 25/87: Detection of discrete points within a voice signal
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 40/44: Statistical methods, e.g. probability models
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/08: Learning methods
- G10L 15/183: using context dependencies, e.g. language models
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method is provided. The method includes receiving a speech input in a first language from a first device; obtaining, by using an artificial intelligence (AI) model, an estimated interpretation time that indicates a time expected to be required to interpret the speech input in the first language into a second language; transmitting, based on the estimated interpretation time, interpretation situation information to at least one of the first device or a second device; interpreting the speech input in the first language into the second language; and transmitting, to the second device a result of the interpreting of the speech input into the second language.

## Claims

### Claim 1

A method performed by a server connected to a first device, a second device and a third device, the method comprising:
receiving a speech input in a first language from the first device;
obtaining, by using an artificial intelligence (AI) model, a first estimated interpretation time that indicates a first time expected to be required to interpret the speech input in the first language into a second language;
transmitting, based on the first estimated interpretation time, interpretation situation information to at least one of the first device, the second device or the third device;
interpreting the speech input in the first language into the second language and a third language;
transmitting, to the second device a result of the interpreting of the speech input into the second language;
transmitting, to the third device a result of the interpreting of the speech input into the third language; and
when the interpreting of the speech input into the third language is not finished, transmitting the interpretation situation information to the second device based on an accumulated delay time.

### Claim 2 (depends on 1)

The method of claim 1, wherein the transmitting of the interpretation situation information comprises:
updating the accumulated delay time based on the first estimated interpretation time; and
identifying whether to transmit the interpretation situation information based on the accumulated delay time.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
measuring an actual interpretation time from a first time point when a speaker's utterance for the speech input is finished to a second time point when reproduction of the result of the interpreting of the speech input into the second language is finished; and
modifying the accumulated delay time based on a difference between the first estimated interpretation time and the actual interpretation time.

### Claim 4 (depends on 1)

The method of claim 1, wherein the transmitting of the interpretation situation information comprises:
increasing the accumulated delay time by the first estimated interpretation time; and
transmitting the interpretation situation information to the first device based on the accumulated delay time being greater than or equal to a threshold value.

### Claim 5 (depends on 4)

The method of claim 4, further comprising modifying the threshold value changes based on at least one of environment information, user information, or utterance information.

### Claim 6 (depends on 1)

The method of claim 1, wherein the AI model comprises a model trained on a plurality of speech inputs in the first language and corresponding times taken to interpret the plurality of speech inputs in the first language into the second language.

### Claim 7 (depends on 1)

The method of claim 1, wherein the AI model comprises an interpretation model configured to interpret the speech input in the first language into the second language,
wherein the interpreting comprises using the interpretation model to interpret the speech input in the first language into the second language, and
wherein the method further comprises:
providing the first estimated interpretation time to the second device; and
providing the result of the interpreting of the speech input into the second language to the second device.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first estimated interpretation time indicates an expected time from a first time point when a speaker's utterance for the speech input is finished to a second time point when reproduction of the result of the interpreting of the speech input into the second language is finished.

### Claim 9 (depends on 1)

The method of claim 1, wherein the interpretation situation information comprises at least one of information about whether interpretation is delayed, a command for instructing an output of a notification of an interpretation situation, information about a device from which the notification of the interpretation situation is to be output, information about the accumulated delay time, information about a language for which the interpretation is delayed, or a message notifying that the interpretation is delayed.

### Claim 10 (depends on 1)

The method of claim 1, wherein the method further comprises:
measuring an actual interpretation time required from a first time point when a speaker's utterance for the speech input is finished to a second time point when reproduction of the result of the interpreting of the speech input into the second language is finished; and
decreasing or increasing the accumulated delay time by a difference between the first estimated interpretation time and the actual interpretation time required.

### Claim 11

A server comprising:
a communication interface configured to communicate with a plurality of devices, the plurality of devices comprising a first device, a second device and a third device;
a memory storing one or more instructions; and
a processor configured to execute the one or more instructions stored in the memory to:
receive, via the communication interface, a speech input in a first language from the first device;
obtain, by using an artificial intelligence (AI) model, a first estimated interpretation time that indicates a first time expected to be required to interpret the speech input in the first language into a second language;
transmit, via the communication interface, interpretation situation information to at least one of the first device, the second device or the third device based on the first estimated interpretation time;
interpret the speech input in the first language into the second language and a third language;
transmit, via the communication interface, a result of the interpreting of the speech input into the second language to the second device;
transmit, via the communication interface, a result of the interpreting of the speech input into the third language to the third device; and
when interpretation of the speech input into the third language is not finished, transmit, via the communication interface, the interpretation situation information to the second device based on an accumulated delay time.

### Claim 12 (depends on 11)

The server of claim 11, wherein the processor is further configured to execute the one or more instructions to:
update the accumulated delay time based on the first estimated interpretation time; and
identify whether to transmit the interpretation situation information, based on the accumulated delay time.

### Claim 13 (depends on 11)

The server of claim 11, wherein the processor is further configured to execute the one or more instructions to:
increase the accumulated delay time by the first estimated interpretation time; and
transmit, via the communication interface, the interpretation situation information to the first device based on the accumulated delay time being greater than or equal to a threshold value.

### Claim 14 (depends on 11)

The server of claim 11, wherein the processor is further configured to execute the one or more instructions to:
after transmitting the result of the interpreting of the speech input into the second language to the second device, measure an actual interpretation time required from a first time point when a speaker's utterance for the speech input is finished to a second time point when reproduction of the result of the interpreting of the speech input into the second language is finished; and
decrease or increase the accumulated delay time by a difference between the first estimated interpretation time and the actual interpretation time required.

### Claim 15

A method performed by a first device, the method comprising:
receiving a speech input in a first language;
transmitting the speech input in the first language to a second device and a third device;
receiving, from the second device, a first estimated interpretation time that indicates a first time expected to be required to interpret the speech input in the first language into a second language;
receiving, from the third device, a second estimated interpretation time that indicates a second time expected to be required to interpret the speech input in the first language into a third language; and
providing interpretation situation information to at least one of the first device or the second device based on the first estimated interpretation time; and
when the interpreting of the speech input into the third language is not finished, transmitting the interpretation situation information to the second device based on an accumulated delay time.