# US 2019/0378508 A1 · Information processing device, information processing system, and information processing method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378508 A1
- Document ID: US20190378508A1
- Status: Published application
- Kind: A1
- Application number: 16/477,289
- Filed: 2018-10-05
- Published: 2019-12-12
- Earliest claimed: 2017-12-22
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Keisuke TOUYAMA
- Web page: https://patentgrep.com/patent/US20190378508A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/16: Sound input; Sound output (speech processing G10L)
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L 15/10: using distance or distortion measures between unknown speech and reference templates
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

When a confirmation response to a user speech is generated, a confirmation response capable of reducing a possibility of the occurrence of misunderstanding by adding additional information acquired from dictionary data or an application execution results is generated. Provided is an information processing device including an audio input unit that receives a user speech, a confirmation response generating unit that generates a confirmation response for requesting confirmation of a speaking user to the user speech, and an audio output unit that outputs the confirmation response generated by the confirmation response generating unit, in which the confirmation response generating unit executes generation of a confirmation response using additional information acquired from at least any of dictionary data and an application execution result. For example, the confirmation response is generated using general knowledge base dictionary data, an application execution result, or information obtained from a captured image, a sensor, or the like.

## Claims

### Claim 1

An information processing device, comprising:
an audio input unit that receives a user speech;
a confirmation response generating unit that generates a confirmation response for requesting confirmation of a speaking user to the user speech; and
an audio output unit that outputs the confirmation response generated by the confirmation response generating unit,
wherein the confirmation response generating unit executes generation of a confirmation response using additional information acquired from at least any of dictionary data and an application execution result.

### Claim 2 (depends on 1)

The information processing device according to claim 1,
wherein the confirmation response generating unit executes the generation of the confirmation response using additional information configured with general knowledge obtained from general knowledge base dictionary data.

### Claim 3 (depends on 2)

The information processing device according to claim 2, wherein the general knowledge is at least any of reading information with a date and calendar information.

### Claim 4 (depends on 2)

The information processing device according to claim 2, wherein the general knowledge is at least any of history information and organization information.

### Claim 5 (depends on 1)

The information processing device according to claim 1,
wherein the confirmation response generating unit executes the generation of the confirmation response using information obtained from an application executed in an application executing unit.

### Claim 6 (depends on 5)

The information processing device according to claim 5,
wherein the application executed in the application executing unit is a schedule registration application, and
the confirmation response generating unit executes the generation of the confirmation response using schedule information of a speaking user obtained from the schedule registration application.

### Claim 7 (depends on 5)

The information processing device according to claim 5,
wherein the application executed in the application executing unit is an acquaintance registration application, and
the confirmation response generating unit executes the generation of the confirmation response using acquaintance information of a speaking user obtained from the acquaintance registration application.

### Claim 8 (depends on 1)

The information processing device according to claim 1, further comprising:
an image input unit that captures a peripheral image of the speaking user or a sensor that inputs peripheral information of the speaking user,
wherein the confirmation response generating unit analyzes an input image from the image input unit or sensor detection information and generates a confirmation response using information obtained from an analysis result.

### Claim 9 (depends on 8)

The information processing device according to claim 8,
wherein the confirmation response generating unit identifies a person or an object included in the input image from the image input unit and generates a confirmation response using an identification result.

### Claim 10 (depends on 8)

The information processing device according to claim 8,
wherein the confirmation response generating unit identifies an attribute of a person included in the input image from the image input unit and executes the generation of the confirmation response corresponding to the identified attribute.

### Claim 11 (depends on 8)

The information processing device according to claim 8,
wherein the confirmation response generating unit identifies an age of a person included in the input image from the image input unit and executes the generation of the confirmation response corresponding to the identified age.

### Claim 12 (depends on 8)

The information processing device according to claim 8,
wherein the confirmation response generating unit identifies a configuration of the person included in the input image from the image input unit and executes the generation of the confirmation response corresponding to the identified configuration of the person.

### Claim 13 (depends on 1)

The information processing device according to claim 1,
wherein the confirmation response generating unit generates an image for the confirmation response in conjunction with an audio output of the confirmation response and outputs the image to an image output unit.

### Claim 14 (depends on 1)

The information processing device according to claim 1,
wherein the confirmation response generating unit generates a device control signal for the confirmation response in conjunction with an audio output of the confirmation response and outputs the device control signal to an external device to be controlled.

### Claim 15 (depends on 1)

The information processing device according to claim 1,
wherein the confirmation response generating unit executes a confirmation response generation process based on conversation history information.

### Claim 16 (depends on 1)

The information processing device according to claim 1, further comprising:
a voice recognizing unit that executes a recognition process of an input audio of the audio input unit;
an intention understanding unit that executes an intention understanding process based on a recognition result of the voice recognizing unit; and
a conversation control unit that executes a conversation control process based on an intention understanding result generated by the intention understanding unit,
wherein the conversation control unit determines whether or not a confirmation response for requesting confirmation of a speaking user to the user speech is given on a basis of the intention understanding result generated by the intention understanding unit, and
the confirmation response generating unit executes a confirmation response generation process in a case in which the conversation control unit determines that the confirmation response is given.

### Claim 17

An information processing system, comprising
a user terminal; and
a data processing server,
wherein the user terminal includes
an audio input unit that receives a user speech,
a confirmation response generating unit that generates a confirmation response for requesting confirmation of a speaking user to the user speech, and
an audio output unit that outputs the confirmation response generated by the confirmation response generating unit,
the data processing server includes
a voice recognizing unit that executes a recognition process of an input audio of the audio input unit,
an intention understanding unit that executes an intention understanding process based on a recognition result of the voice recognizing unit, and
a conversation control unit that executes a conversation control process based on an intention understanding result generated by the intention understanding unit, determines whether or not a confirmation response to the user speech is necessary on a basis of the intention understanding result, and outputs a confirmation response generation command to the confirmation response generating unit of the user terminal in a case in which it is determined that the confirmation response is necessary, and
the confirmation response generating unit of the user terminal executes generation of a confirmation response using additional information acquired from at least any of dictionary data and an application execution result in accordance with an input of the confirmation response generation command from the data processing server.

### Claim 18

An information processing method executed in an information processing device, comprising:
an audio input step of receiving, by an audio input unit, a user speech;
a confirmation response generation step of generating, by a confirmation response generating unit, a confirmation response for requesting confirmation of a speaking user to the user speech; and
an audio output step of outputting, by an audio output unit, the confirmation response generated in the confirmation response generation step,
wherein, in the confirmation response generation step, generation of a confirmation response using additional information acquired from at least any of dictionary data and an application execution result is executed.

### Claim 19

An information processing method executed in an information processing system including a user terminal and a data processing server, comprising:
executing, by the user terminal, an audio input process of receiving a user speech;
executing, by the data processing server, an audio recognition process of an input voice input in the audio input process;
executing, by the data processing server, an intention understanding process based on a voice recognition result generated in the voice recognition process;
determining, by the data processing server, whether or not a confirmation response to the user speech is necessary on a basis of an intention understanding result of the intention understanding process, and outputting a confirmation response generation command to the user terminal in a case in which it is determined that the confirmation response is necessary; and
executing, by the user terminal, a confirmation response generation process of generating a confirmation response for requesting confirmation of a speaking user to the user speech in accordance with an input of the confirmation response generation command from the data processing server,
wherein, in the confirmation response generation process, a confirmation response generation process using additional information obtained from at least any of dictionary data and an application execution result is executed.

### Claim 20

A program causing information processing to be executed in an information processing device, comprising:
an audio input step of causing an audio input unit to receive a user speech;
a confirmation response generation step of causing a confirmation response generating unit to generate a confirmation response for requesting confirmation of a speaking user to the user speech; and
an audio output step of causing an audio output unit to output the confirmation response,
wherein, in the confirmation response generation step, generation of a confirmation response using additional information acquired from at least any of dictionary data and an application execution result is executed.
