# US 2024/0203408 A1 · Speech Wakeup Method and Apparatus, Device, Storage Medium, and Program Product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203408 A1
- Document ID: US20240203408A1
- Status: Published application
- Kind: A1
- Application number: 18/591,853
- Filed: 2024-02-29
- Published: 2024-06-20
- Earliest claimed: 2021-08-30
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Xiaojian Li; Jinhui Sheng; Yue Lang; Wei Jiang
- Web page: https://patentgrep.com/patent/US20240203408A1

## CPC

- G10L 15/187: Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 15/24: Speech recognition using non-acoustical features
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- G10L 2015/025: {Phonemes, fenemes or fenones being the recognition units}
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A speech wakeup method includes that a bone conduction microphone collects a bone conduction signal for speech detection, which includes information about a command word input by the sound source, and detects a wakeup word based on the bone conduction signal.

## Claims

### Claim 1

A method comprising:
collecting, using a bone conduction microphone, a bone conduction signal from a sound source, wherein the bone conduction signal comprises information about a command word;
obtaining a fusion signal based on the bone conduction signal;
performing speech detection based on the bone conduction signal to attempt to detect a speech input;
detecting, when the speech input is detected, detecting whether the command word comprises a wakeup word based on the fusion signal; and
performing, when the command word comprises the wakeup word, speech wakeup on a to-be-woken-up device.

### Claim 2

(canceled)

### Claim 3 (depends on 1)

The method according to claim 1, wherein before the obtaining the fusion signal, the method further comprises:
turning on an air microphone; and
collecting an air conduction signal using the air microphone, and
wherein obtaining the fusion signal comprises:
detecting a start part of the bone conduction signal based on a detection delay when performing the speech detection, and fusing the start part and the air conduction signal to obtain the fusion signal;
detecting a start part of the bone conduction signal based on a detection delay when performing the speech detection, generating an enhancement start signal based on the start part, and fusing the enhancement start signal and the air conduction signal to obtain the fusion signal; or
directly fusing the bone conduction signal and the air conduction signal to obtain the fusion signal.

### Claim 4 (depends on 1)

The method according to claim 1, wherein detecting the wakeup word comprises:
inputting a plurality of audio frames from the fusion signal into an acoustic model;
obtaining a plurality of posterior probability vectors by the acoustic model, wherein the posterior probability vectors are in a one-to-one correspondence with the audio frames, and wherein a first posterior probability vector from the posterior probability vectors indicates a probability that a first phoneme of a first audio frame from the audio frames belongs to a plurality of pre-indicated phonemes; and
detecting the wakeup word based on the posterior probability vectors.

### Claim 5 (depends on 1)

The method according to claim 1, wherein before detecting the wakeup word, the method further comprises:
turning on an air microphone; and
collecting an air conduction signal using the air microphone, and wherein detecting the wakeup word based on the bone conduction signal comprises:
obtaining a plurality of posterior probability vectors based on the bone conduction signal and the air conduction signal, wherein the posterior probability vectors are in a one-to-one correspondence with a plurality of audio frames from the bone conduction signal and the air conduction signal, and wherein a first posterior probability vector from the posterior probability vectors indicates a probability that a first phoneme of a first audio frame from the audio frames belongs to a plurality of pre-indicated phonemes; and
detecting the wakeup word based on the plurality of posterior probability vectors.

### Claim 6 (depends on 5)

The method according to claim 5, wherein obtaining the posterior probability vectors comprises:
detecting a start part of the bone conduction signal based on a detection delay when performing the speech detection;
inputting a start part of the bone conduction signal and the air conduction signal to an acoustic model;
obtaining a plurality of bone conduction posterior probability vectors and a plurality of air conduction posterior probability vectors by the acoustic model, wherein the bone conduction posterior probability vectors are in a one-to-one correspondence with a plurality of first audio frames from the start part of the bone conduction signal, and wherein the air conduction posterior probability vectors are in a one-to-one correspondence with a plurality of second audio frames from the air conduction signal; and
fusing a first bone conduction posterior probability vector and a first air conduction posterior probability vector to obtain a second posterior probability vector, wherein the first bone conduction posterior probability vector corresponds to a last audio frame from the start part of the bone conduction signal, and wherein a last audio frame has a second duration less than a frame duration.

### Claim 7 (depends on 5)

The method according to claim 5, wherein obtaining the posterior probability vectors comprises:
detecting a start part of the bone conduction signal based on a detection delay when performing the speech detection;
inputting a start part of the bone conduction signal and the air conduction signal to an acoustic model; and
obtaining the posterior probability vectors by the acoustic model.

### Claim 8 (depends on 4)

The method according to claim 4, wherein detecting the wakeup word comprises:
determining, based on the posterior probability vectors and a first phoneme sequence corresponding to the wakeup word, a confidence level that a second phoneme sequence corresponding to the command word comprises the first phoneme sequence; and
detecting, when the confidence level exceeds a confidence threshold, that the command word comprises the wakeup word.

### Claim 9 (depends on 4)

The method according to claim 4, wherein detecting the wakeup word comprises:
determining, based on the posterior probability vectors and a first phoneme sequence corresponding to the wakeup word, a confidence level that a first phoneme sequence corresponding to the command word comprises the first phoneme sequence; and
detecting, when the confidence level exceeds a confidence threshold and when the posterior probability vectors and a plurality of template vectors meet a distance condition, that the command word comprises the wakeup word, wherein the template vectors indicate a probability that a second phoneme of a speech signal comprising information of the wakeup word belongs to the pre-indicated phonemes.

### Claim 10 (depends on 9)

The method according to claim 9, wherein when the posterior probability vectors are in a one-to-one correspondence with the template vectors, the distance condition comprises that a mean of distances between the posterior probability vectors and the template vectors is less than a distance threshold.

### Claim 11 (depends on 9)

The method according to claim 9, wherein the method further comprises:
obtaining a bone conduction registration signal, wherein the bone conduction registration signal comprises the information of the wakeup word; and
determining the confidence threshold and the template vectors based on the bone conduction registration signal and the first phoneme sequence.

### Claim 12 (depends on 11)

The method according to claim 11, wherein determining the confidence threshold and the plurality of template vectors comprises:
obtaining a fusion registration signal based on the bone conduction registration signal; and
determining the confidence threshold and the plurality of template vectors based on the fusion registration signal and the first phoneme sequence.

### Claim 13 (depends on 12)

The method according to claim 12, wherein determining the confidence threshold and the plurality of template vectors further comprises:
inputting a plurality of registration audio frames from the fusion registration signal into the acoustic model; and
obtaining a plurality of registration posterior probability vectors by the acoustic model, wherein the registration posterior probability vectors are in a one-to-one correspondence with the registration audio frames, and wherein a first registration posterior probability vector from the registration posterior probability vectors indicates a probability that a third phoneme of a first registration audio frame from the registration audio frames belongs to pre-indicated phonemes;
using the registration posterior probability vectors as the template vectors; and
determining the confidence threshold based on the registration posterior probability vectors and the first phoneme sequence.

### Claim 14 (depends on 11)

The method according to claim 11, wherein before determining the confidence threshold, the method further comprises obtaining an air conduction registration signal, and wherein determining the confidence threshold comprises:
determining a plurality of registration posterior probability vectors based on the bone conduction registration signal and the air conduction registration signal, wherein the registration posterior probability vectors are in a one-to-one correspondence with a plurality of registration audio frames from the bone conduction registration signal and the air conduction registration signal, and wherein a first registration posterior probability vector from the registration posterior probability vectors indicates a probability that a second phoneme of a first registration audio frame from the registration audio frames belongs to the pre-indicated phonemes; and
determining the confidence threshold based on the registration posterior probability vectors and the first phoneme sequence.

### Claim 15

An electronic device, wherein the electronic device comprises
a memory configured to store a computer program; and
at least one processor coupled to the memory and configured to execute the computer program to cause the electronic device to:
collect, using a bone conduction microphone, a bone conduction signal from a sound source, wherein the bone conduction signal comprises information about a command word;
obtain a fusion signal based on the bone conduction signal;
perform speech detection based on the bone conduction signal to attempt to detect a speech input;
detect, when the speech input is detected, whether the command word comprises a wakeup word based on the fusion signal; and
perform, when the command word comprises the wakeup word, speech wakeup on a to-be-woken-up device.

### Claim 16

(canceled)

### Claim 17 (depends on 15)

The electronic device according to claim 15, wherein before obtaining the fusion signal, the at least one processor is further configured to execute the computer program to cause the electronic device to:
turn on an air microphone; and
collect an air conduction signal using the air microphone, and wherein the at least one processor is further configured to execute the computer program to cause the electronic device to:
detect a start part of the bone conduction signal based on a detection delay when performing the speech detection, and fuse the start part and the air conduction signal to obtain the fusion signal;
detect a start part of the bone conduction signal based on a detection delay when performing the speech detection, and generate an enhancement start signal based on the start part, and fuse the enhancement start signal and the air conduction signal to obtain the fusion signal; or
fuse the bone conduction signal and the air conduction signal to obtain the fusion signal.

### Claim 18 (depends on 15)

The electronic device according to claim 15, wherein the at least one processor is further configured to execute the computer program to cause the electronic device to:
inputting, a plurality of audio frames from the fusion signal into an acoustic model;
obtain a plurality of posterior probability vectors using the acoustic model, wherein the posterior probability vectors are in a one-to-one correspondence with the audio frames, and wherein a first posterior probability vector from the posterior probability vectors indicates a probability that a first phoneme of a first audio frame from the audio frames belongs to a plurality of pre-indicated phonemes; and
detect the wakeup word based on the posterior probability vectors.

### Claim 19

(canceled)

### Claim 20

A computer program comprising computer instructions, that are stored on a non-transitory computer-readable storage medium and that, when executed by at least one processor, cause an electronic device to:
collect, using a bone conduction microphone, a bone conduction signal from a sound source, wherein the bone conduction signal comprises information about a command word;
perform speech detection based on a bone conduction signal to attempt to detect a speech input;
detect, when the speech input is detected, whether the command word comprises a wakeup word based on the bone conduction signal; and
perform, when the command word comprises the wakeup word, speech wakeup on a to-be-woken-up device.

### Claim 21 (depends on 5)

The method according to claim 5, wherein obtaining the posterior probability vectors comprises:
inputting the bone conduction signal and the air conduction signal to an acoustic model; and
obtaining the posterior probability vectors by the acoustic model.

### Claim 22 (depends on 6)

The method according to claim 6, wherein the first air conduction posterior probability vector corresponds to a first audio frame of the air conduction signal, wherein the first audio frame has a first duration less than the frame duration, and wherein the posterior probability vectors comprise the second posterior probability vector, a first vector from the bone conduction posterior probability vectors other than the first bone conduction posterior probability vector, and a second vector from the air conduction posterior probability vectors other than the first air conduction posterior probability vector.

### Claim 23 (depends on 18)

The electronic device according to claim 18, wherein the at least one processor is further configured to execute the computer program to cause the electronic device to:
turn on an air microphone coupled to the electronic device or the at least one processor; and
collect an air conduction signal using the air microphone, and
wherein the at least one processor is further configured to execute the computer program to cause the electronic device to:
obtain a plurality of posterior probability vectors based on the bone conduction signal and the air conduction signal, wherein the posterior probability vectors are in a one-to-one correspondence with a plurality of audio frames from the bone conduction signal and the air conduction signal, and wherein a first posterior probability vector from the posterior probability vectors indicates a probability that a first phoneme of a first audio frame from the audio frames belongs to a plurality of pre-indicated phonemes; and
detect the wakeup word based on the posterior probability vectors.
