# US 12,217,767 B2 · Audio device with sound determination

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,767 B2
- Document ID: US12217767B2
- Status: Granted
- Kind: B2
- Application number: 17/630,090
- Filed: 2020-07-29
- Published: 2022-11-17
- Granted: 2025-02-04
- Earliest claimed: 2019-08-09
- Assignees: Semiconductor Energy Laboratory Co., Ltd.
- Applicants: Semiconductor Energy Laboratory Co., Ltd.
- Inventors: Fumiya Nagashima; Kengo Akimoto; Tatsuya Okano; Yusuke Koumura; Seiko Inoue
- Web page: https://patentgrep.com/patent/US12217767B2

## CPC

- G10L 17/06: Decision making techniques; Pattern matching strategies
- G10L 21/0272: Voice signal separating
- G10K 11/17827: {Desired external signals, e.g. pass-through audio such as music or speech}
- G10K 11/17855: {for improving speed or power requirements}
- G10K 11/17873: {using a reference signal without an error signal, e.g. pure feedforward}
- G10L 15/063: {Training}
- G10L 15/16: using artificial neural networks
- G10L 17/04: Training, enrolment or model building
- G10L 25/30: using neural networks
- G10K 2210/1081: Earphones, e.g. for telephones, ear protectors or headsets
- G10K 2210/3024: Expert systems, e.g. artificial intelligence
- G10L 17/18: Artificial neural networks; Connectionist approaches
- G10L 2021/02087: {the noise being separate speech, e.g. cocktail party}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10K: SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR

## Abstract

An audio device capable of inhibiting malfunction of an information terminal is provided. The audio device includes a sound sensor portion, a sound separation portion, a sound determination portion, and a processing portion. The sound sensor portion has a function of sensing sound. The sound separation portion has a function of separating the sound sensed by the sound sensor portion into a voice and sound other than a voice. The sound determination portion has a function of storing the feature quantity of the sound. The sound determination portion has a function of determining, with a machine learning model such as a neural network model, whether the feature quantity of the voice separated by the sound separation portion is the stored feature quantity. The processing portion has a function of analyzing an instruction contained in the voice and generating an instruction signal representing the content of the instruction in the case where the feature quantity of the voice is the stored feature quantity. The processing portion has a function of performing, on the sound other than a voice separated by the sound separation portion, processing for canceling the sound other than a voice. Specifically, the processing portion has a function of performing, on the sound other than a voice, processing for inverting the phase thereof.

## Claims

### Claim 1

An audio device comprising:
a sound sensor portion;
a sound separation portion;
a sound determination portion;
a processing portion; and
an output portion,
wherein the audio device is an earphone or a headphone,
wherein the sound sensor portion is configured to sense first sound,
wherein the sound determination portion is configured to store a feature quantity of a voice of a user,
wherein the sound determination portion is configured to determine, with a machine learning model, whether the first sound has the stored feature quantity,
wherein the processing portion is configured to analyze an instruction contained in the first sound and, when the feature quantity of the first sound is the stored feature quantity, to generate a signal representing content of the instruction, and, when the feature quantity of the first sound is not the stored feature quantity, to not generate the signal representing content of the instruction,
wherein second sound is sound to cancel noise in the first sound,
wherein third sound is sound electrically input to the audio device,
wherein fourth sound is synthesized sound of the second sound and the third sound,
wherein the processing portion is configured to generate the second sound and the third sound, and
wherein the output portion is configured to output the fourth sound.

### Claim 2 (depends on 1)

The audio device according to claim 1,
wherein learning for the machine learning model is performed using supervised learning in which a voice is learning data and a label indicating whether the storing is to be performed is training data.

### Claim 3 (depends on 1)

The audio device according to claim 1,
wherein the machine learning model is a neural network model.

### Claim 4 (depends on 1)

The audio device according to claim 1,
wherein the second sound is sound having a phase opposite to a phase of the noise in the first sound.

### Claim 5

An operation method of an audio device, comprising:
sensing first sound;
determining, with a machine learning model, whether a feature quantity of the first sound is a stored feature quantity of a voice of a user;
analyzing an instruction contained in the first sound;
generating a signal representing content of the instruction when the feature quantity of the first sound is the stored feature quantity and not generating the signal representing content of the instruction when the feature quantity of the first sound is not the stored feature quantity;
generating second sound to cancel noise in the first sound;
electrically inputting third sound to the audio device;
generating fourth sound by synthesizing the second sound and the third sound; and
outputting the fourth sound.

### Claim 6 (depends on 5)

The operation method of the audio device, according to claim 5,
wherein learning for the machine learning model is performed using supervised learning in which a voice is used as learning data and a label indicating whether storing is to be performed is used as training data.

### Claim 7 (depends on 5)

The operation method of the audio device, according to claim 5,
wherein the machine learning model is a neural network model.

### Claim 8 (depends on 5)

The operation method of the audio device, according to claim 5,
wherein the second sound is sound having a phase opposite to a phase of the noise in the first sound.