# US 12,763,068 B2 · Training device, estimation device, training method, estimation method, training program, and estimation program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,763,068 B2
- Document ID: US12763068B2
- Status: Granted
- Kind: B2
- Application number: 18/840,512
- Filed: 2022-02-24
- Published: 2025-05-22
- Granted: 2026-10-13
- Earliest claimed: 2022-02-24
- Assignees: NTT, Inc.
- Applicants: NTT, Inc.
- Inventors: Akira Morikawa; Ryo Ishii; Hajime Noto
- Web page: https://patentgrep.com/patent/US12763068B2

## CPC

- G10L 15/063: {Training}
- G10L 15/08: Speech classification or search
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A learning device includes processing circuitry configured to acquire speech data of a speaker, information on the speaker, conversation data of a listener, information on the listener, and a classification label of a quick response included in the conversation data of the listener, and create a learned model of estimating a type of the quick response of the listener to a conversation of the speaker using the information acquired with the classification label of the quick response as correct answer data.

## Claims

### Claim 1

A learning device comprising:
processing circuitry configured to:
obtain first multimodal data of utterances, wherein the first multimodal data includes one or more expressions, actions, and voices of speakers;
obtain second multimodal data of backchannel responses, wherein the second multimodal data includes one or more expressions, actions, and voices of listeners;
obtain classification data including classification labels associated with the backchannel responses, wherein each classification label indicates one of (i) a positive response, (ii) a neutral response, (iii) a non-positive response, (iv) an emotional word, (v) a confirmation, (vi) a repetition of a speaker's utterance, (vii) a response providing a topic, and (viii) a summarization or paraphrase of a speaker's utterance; and
train, using the first multimodal data, the second multimodal data, and the classification data, a model, wherein the first and second multimodal data are used as training input, and the classification data is used as training output.

### Claim 2 (depends on 1)

The learning device according to claim 1, wherein the backchannel responses include at least one of “yes, yes, yes,” “yeah, yeah,” “good,” “I see,” “true,” “yeah,” “yes,” “oh,” “ooh,” “hmm,” “great,” and “what”.

### Claim 3

A learning method performed by a learning device, the learning method comprising:
obtaining first multimodal data of utterances, wherein the first multimodal data includes one or more expressions, actions, and voices of speakers;
obtaining second multimodal data of backchannel responses, wherein the second multimodal data includes one or more expressions, actions, and voices of listeners;
obtaining classification data including classification labels associated with the backchannel responses, wherein each classification label indicates one of (i) a positive response, (ii) a neutral response, (iii) a non-positive response, (iv) an emotional word, (v) a confirmation, (vi) a repetition of a speaker's utterance, (vii) a response providing a topic, and (viii) a summarization or paraphrase of a speaker's utterance; and
training, using the first multimodal data, the second multimodal data, and the classification data, a model, wherein the first and second multimodal data are used as training input, and the classification data is used as training output.

### Claim 4 (depends on 3)

The learning method according to claim 3, wherein the backchannel responses include at least one of “yes, yes, yes,” “yeah, yeah,” “good,” “I see,” “true,” “yeah,” “yes,” “oh,” “ooh,” “hmm,” “great,” and “what”.

### Claim 5

A non-transitory computer-readable recording medium storing therein a learning program that causes a computer to execute a process comprising:
obtaining first multimodal data of utterances, wherein the first multimodal data includes one or more expressions, actions, and voices of speakers;
obtaining second multimodal data of backchannel responses, wherein the second multimodal data includes one or more expressions, actions, and voices of listeners;
obtaining classification data including classification labels associated with the backchannel responses, wherein each classification label indicates one of (i) a positive response, (ii) a neutral response, (iii) a non-positive response, (iv) an emotional word, (v) a confirmation, (vi) a repetition of a speaker's utterance, (vii) a response providing a topic, and (viii) a summarization or paraphrase of a speaker's utterance; and
training, using the first multimodal data, the second multimodal data, and the classification data, a model, wherein the first and second multimodal data are used as training input, and the classification data is used as training output.

### Claim 6 (depends on 5)

The non-transitory computer-readable recording medium according to claim 5, wherein the backchannel responses include at least one of “yes, yes, yes,” “yeah, yeah,” “good,” “I see,” “true,” “yeah,” “yes,” “oh,” “ooh,” “hmm,” “great,” and “what”.