# US 2024/0202634 A1 · Dialogue training device, dialogue training system, dialogue training method, and computer-readable medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202634 A1
- Document ID: US20240202634A1
- Status: Published application
- Kind: A1
- Application number: 18/536,267
- Filed: 2023-12-12
- Published: 2024-06-20
- Earliest claimed: 2022-12-14
- Inventors: Jun NAKAGAWA; Haruhiko YASUDA
- Web page: https://patentgrep.com/patent/US20240202634A1

## CPC

- G06Q 10/0639: Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06V 40/174: {Facial expression recognition}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G10L 15/08: Speech classification or search
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 25/63: for estimating an emotional state
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A dialogue training device includes an obtaining unit and a first analyzing unit. The obtaining unit is configured to, during a dialogue in which a voice of an answer to voice data of an utterance made by a user is output using a video of a virtual person generated based on information related to a target person assumed to conduct the dialogue, obtain the voice data and video data of the user input into a terminal device. The first analyzing unit is configured to, based on the voice data and the video data obtained by the obtaining unit, analyze a dialogue skill with the virtual person.

## Claims

### Claim 1

A dialogue training device comprising:
an obtaining unit configured to, during a dialogue in which a voice of an answer to voice data of an utterance made by a user is output using a video of a virtual person generated based on information related to a target person assumed to conduct the dialogue, obtain the voice data and video data of the user input into a terminal device; and
a first analyzing unit configured to, based on the voice data and the video data obtained by the obtaining unit, analyze a dialogue skill with the virtual person.

### Claim 2 (depends on 1)

The dialogue training device according to claim 1, wherein the first analyzing unit includes:
an image analyzing unit configured to analyze behavior of the user with respect to the virtual person from the video data obtained by the obtaining unit, and analyze the dialogue skill based on the behavior; and
a voice analyzing unit configured to analyze an answer given by the user to the virtual person from the voice data obtained by the obtaining unit, and analyze the dialogue skill based on the answer.

### Claim 3 (depends on 2)

The dialogue training device according to claim 2, wherein
the image analyzing unit is configured to analyze, as the behavior, at least any of a number of times nodding to the virtual person is performed, and a number of times smiling at the virtual person is performed, and
the voice analyzing unit is configured to analyze, as the answer, at least any of a number of times a positive reception to the virtual person is made, a number of times a concern for the virtual person is made, a number of times a supportive response to the virtual person is made, a number of times an open question to the virtual person is made, and an utterance ratio during the dialogue with the virtual person.

### Claim 4 (depends on 1)

The dialogue training device according to claim 1, wherein the first analyzing unit is configured to, based on the voice data and the video data obtained by the obtaining unit, analyze at least any of an attentive listening skill, an acknowledgment skill, and a questioning skill as the dialogue skill.

### Claim 5 (depends on 1)

The dialogue training device according to claim 1, further comprising an output unit configured to cause the terminal device to display an analysis result by the first analyzing unit.

### Claim 6 (depends on 5)

The dialogue training device according to claim 5, wherein
the first analyzing unit is configured to quantify the dialogue skill, and
the output unit is configured to cause the terminal device to display a value of the dialogue skill quantified by the first analyzing unit, as the analysis result.

### Claim 7 (depends on 5)

The dialogue training device according to claim 5, wherein the output unit is configured to:
convert the voice data into a text; and
cause the terminal device to display the text in which a portion having contribution from the dialogue skill is highlighted, as the analysis result.

### Claim 8 (depends on 1)

The dialogue training device according to claim 1, further comprising a second analyzing unit configured to, based on the voice data and the video data obtained by the obtaining unit, analyze an emotion of the user during the dialogue with the virtual person, wherein
the obtaining unit is configured to obtain the voice data and the video data during the dialogue with the virtual person using the video generated for outputting the voice of the answer to the voice data based on the voice data and the emotion of the user.

### Claim 9 (depends on 1)

A dialogue training system comprising:
the dialogue training device according to claim 1;
a first generation device configured to, based on information related to the target person, generate data of the virtual person mapped to the target person; and
a second generation device configured to, based on the data of the virtual person generated by the first generation device, generate a video of the virtual person.

### Claim 10

A dialogue training method comprising:
during a dialogue in which a voice of an answer to voice data of an utterance made by a user is output using a video of a virtual person generated based on information related to a target person assumed to conduct the dialogue, obtaining the voice data and video data of the user input into a terminal device; and
analyzing a dialogue skill with the virtual person based on the obtained voice data and the obtained video data.

### Claim 11

A non-transitory computer-readable medium including programmed instructions that cause a computer to execute:
during a dialogue in which a voice of an answer to voice data of an utterance made by a user is output using a video of a virtual person generated based on information related to a target person assumed to conduct the dialogue, obtaining the voice data and video data of the user input into a terminal device; and
analyzing a dialogue skill with the virtual person based on the obtained voice data and the obtained video data.
