US 12,119,017 B2Granted

Information processing device, information processing system and information processing method

AssigneeSony Group Corporation

InventorsAkira Takahashi, Kazuya Tateishi, Yuichiro Koyama, Hiroaki Ogawa, Chie Kamada, Noriko Totsuka, Emiro Tsunoo, Yuki Takeda, Yoshinori Maeda, Kan Kuroda, Akira Fukui, Hideaki Watanabe

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Provided is a device that includes a user spoken voice extraction unit that extracts a user spoken voice from a microphone input sound. The user spoken voice extraction unit analyzes a sound source direction of an input sound, determines whether the input sound includes an external apparatus output sound on the basis of sound source directions of external apparatus output sounds recorded in a database, and removes a sound signal corresponding to a feature amount, for example, a frequency characteristic of the external apparatus output sound recorded in the database, from the input sound to extract a user spoken voice from which the external apparatus output sound has been removed upon determining that the input sound includes the external apparatus output sound.

Claims13

Dependent claims

An information processing device, comprising:

an external apparatus output sound characteristic database that includes sound source directions of a plurality of external apparatus output sounds, wherein each sound source direction of the sound source directions corresponds to a respective external apparatus output sound of the plurality of external apparatus output sounds;
and
a central processing unit (CPU) configured to:
analyze a plurality of characteristics of an external apparatus output sound of an external apparatus, wherein
the plurality of characteristics of the external apparatus output sound includes a first sound source direction of the external apparatus output sound and a first frequency characteristic of the external apparatus output sound,
the plurality of external apparatus output sounds includes the external apparatus output sound, and
the sound source directions include the first sound source direction;
record the analyzed plurality of characteristics of the external apparatus output sound in the external apparatus output sound characteristic database;
cause output of audio data having a second frequency characteristic from the external apparatus;
receive an input sound that is acquired by a microphone array;
execute analysis of the input sound;
and
analyze, based on the executed analysis of the input sound, a second sound source direction of the input sound and a third frequency characteristic of the input sound, wherein the sound source directions include the second sound source direction;
extract a user spoken voice from the input sound acquired by the microphone array;
determine that the input sound includes the external apparatus output sound based on the sound source directions of the plurality of external apparatus output sounds;
and
remove the external apparatus output sound from the input sound based on
a feature amount of the external apparatus output sound recorded in the external apparatus output sound characteristic database, and
the determination that the input sound includes the external apparatus output sound.

16 more elements

A non-transitory computer-readable medium having stored thereon, computer executable instructions, which when executed by a computer, cause the computer to execute operations, the operations comprising:

analyzing a plurality of characteristics of an external apparatus output sound of an external apparatus, wherein the plurality of characteristics of the external apparatus output sound includes a first sound source direction of the external apparatus output sound and a first frequency characteristic of the external apparatus output sound;
recording the analyzed plurality of characteristics of the external apparatus output sound in an external apparatus output sound characteristic database, wherein
the external apparatus output sound characteristic database includes sound source directions of a plurality of external apparatus output sounds,
each sound source direction of the sound source directions corresponds to a respective external apparatus output sound of the plurality of external apparatus output sounds,
the plurality of external apparatus output sounds includes the external apparatus output sound, and
the sound source directions include the first sound source direction;
causing output of audio data having a second frequency characteristic from the external apparatus;
receiving an input sound that is acquired by a microphone array;
executing analysis of the input sound;
and
analysing, based on the executed analysis of the input sound, a second sound source direction of the input sound and a third frequency characteristic of the input sound, wherein the sound source directions include the second sound source direction;
extracting a user spoken voice from the input sound acquired by the microphone array;
determining the input sound includes the external apparatus output sound based on the sound source directions of the plurality of external apparatus output sounds;
and
removing the external apparatus output sound from the input sound based on
a feature amount of the external apparatus output sound recorded in the external apparatus output sound characteristic database, and
the determination that the input sound includes the external apparatus output sound.

14 more elements

An information processing method, comprising:

analyzing a plurality of characteristics of an external apparatus output sound of an external apparatus, wherein the plurality of characteristics of the external apparatus output sound includes a first sound source direction of the external apparatus output sound and a first frequency characteristic of the external apparatus output sound;
recording the analyzed plurality of characteristics of the external apparatus output sound in an external apparatus output sound characteristic database, wherein
the external apparatus output sound characteristic database includes sound source directions of a plurality of external apparatus output sounds,
each sound source direction of the sound source directions corresponds to a respective external apparatus output sound of the plurality of external apparatus output sounds,
the plurality of external apparatus output sounds includes the external apparatus output sound, and
the sound source directions include the first sound source direction;
causing output of audio data having a second frequency characteristic from the external apparatus;
receiving an input sound that is acquired by a microphone array;
executing analysis of the input sound;
and
analysing, based on the executed analysis of the input sound, a second sound source direction of the input sound and a third frequency characteristic of the input sound, wherein the sound source directions include the second sound source direction;
extracting a user spoken voice from the input sound acquired by the microphone array;
determining the input sound includes the external apparatus output sound based on the sound source directions of the plurality of external apparatus output sounds;
and
removing the external apparatus output sound from the input sound based on
a feature amount of the external apparatus output sound recorded in the external apparatus output sound characteristic database, and
the determination that the input sound includes the external apparatus output sound.

14 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/609,314Filed 25 Mar 2020
Priority: JP 2019-09113114 May 2019
Published: US 2022/0223167 A114 Jul 2022
Granted: 15 Oct 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification11 codes

G10L 21/0232
Characterised by the method used for estimating noise: processing in the frequency domain
H04R 1/326
For obtaining desired directional characteristic only: for microphones
G10L 15/22
Speech recognition: procedures used during a speech recognition process, e.g. man-machine dialogue
G10L 15/30
Constructional details of speech recognition systems: distributed recognition, e.g. in client-server systems, for mobile phones or network…

Family

US 2022/0223167 A1Pre-grant publication · published 14 Jul 2022
US 12,119,017 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key