# US 12,002,464 B2 · Systems and methods for recognizing a speech of a speaker

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,002,464 B2
- Document ID: US12002464B2
- Status: Granted
- Kind: B2
- Application number: 17/813,367
- Filed: 2022-07-19
- Published: 2022-11-03
- Granted: 2024-06-04
- Earliest claimed: 2018-12-28
- Assignees: RingCentral, Inc.
- Applicants: RingCentral, Inc.
- Inventors: Ilya Vladimirovich Mikhailov
- Web page: https://patentgrep.com/patent/US12002464B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 15/04: Segmentation; Word boundary detection
- G10L 15/16: using artificial neural networks
- G10L 15/24: Speech recognition using non-acoustical features
- G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
- G10L 25/90: Pitch determination of speech signals
- G10L 17/00: Speaker identification or verification techniques
- G10L 25/30: using neural networks
- G10L 25/84: for discriminating voice from noise
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for recognizing speech within a received audio signal includes separating, using a computer-based neural network model, a speech from an audio signal based on a speaker's audio profile, determining a command from the speech, determining, from the audio signal, a first score reflecting a percentage of confidence in determining the command based on a frequency of using the command by the speaker, determining, from the audio signal, a second score reflecting a percentage of importance of the command, and causing the command to be executed if the first score is above a first threshold value and the second score is below a second threshold value.

## Claims

### Claim 1

A computer-implemented method applied to a computing device for recognizing speech within a received audio signal, the method comprising:
separating, using a computer-based neural network model, a speaker's speech from an audio signal based on the speaker's audio profile, the audio signal including the speaker's speech and audio signals associated with speech from other speakers or environmental audio signals;
determining a command from the speech;
determining, from the audio signal, a first score reflecting a percentage of confidence in determining the command based on a frequency of using the command by the speaker;
determining, from the audio signal, a second score reflecting a percentage of importance of the command;
determining whether the first score is above a first threshold value and the second score is below a second threshold value; and
causing the command to be executed at the computing device in response to determining that the first score is above the first threshold value and the second score is below the second threshold value.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
reducing background noise prior to the separating the speaker's speech from the audio signal based on the speaker's audio profile.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the separating the speaker's speech from the audio signal comprises separating the speaker's speech based on one of an audio waveform, a cadence of the speech, a pitch of the speech, loudness of the speech, or vocabulary of the speech.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving video data of the speaker producing the audio signal; and
wherein the separating the speaker's speech comprises separating the speaker's speech based on a correlation of the audio signal and the video data.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving vibration data of the speaker producing the audio signal; and
wherein the separating the speaker's speech comprises separating the speaker's speech based on a correlation of the audio signal and the vibration data.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, further comprising:
prompting the speaker for the audio signal containing the command.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein the prompting the speaker comprises prompting using a synthesized speech.

### Claim 8

A non-transitory, computer-readable medium storing a set of instructions that, when executed by a processor in a computing device, cause:
separating, using a computer-based neural network model, a speaker's speech from an audio signal based on the speaker's audio profile, the audio signal including the speaker's speech and audio signals associated with speech from other speakers or environmental audio signals;
determining a command from the speech;
determining, from the audio signal, a first score reflecting a percentage of confidence in determining the command based on a frequency of using the command by the speaker;
determining, from the audio signal, a second score reflecting a percentage of importance of the command;
determining whether the first score is above a first threshold value and the second score is below a second threshold value; and
causing the command to be executed at the computing device in response to determining that the first score is above the first threshold value and the second score is below the second threshold value.

### Claim 9 (depends on 8)

The non-transitory, computer-readable medium of claim 8, storing further instructions that, when executed by the processor, cause:
reducing background noise prior to the separating the speaker's speech from the audio signal based on the speaker's audio profile.

### Claim 10 (depends on 8)

The non-transitory, computer-readable medium of claim 8, wherein the separating the speaker's speech from the audio signal comprises separating the speaker's speech based on one of an audio waveform, a cadence of the speech, a pitch of the speech, loudness of the speech, or vocabulary of the speech.

### Claim 11 (depends on 8)

The non-transitory, computer-readable medium of claim 8, storing further instructions that, when executed by the processor, cause:
receiving video data of the speaker producing the audio signal; and
wherein the separating the speaker's speech comprises separating the speaker's speech based on a correlation of the audio signal and the video data.

### Claim 12 (depends on 8)

The non-transitory, computer-readable medium of claim 8, storing further instructions that, when executed by the processor, cause:
receiving vibration data of the speaker producing the audio signal; and
wherein the separating the speaker's speech comprises separating the speaker's speech based on a correlation of the audio signal and the vibration data.

### Claim 13 (depends on 8)

The non-transitory, computer-readable medium of claim 8, storing further instructions that, when executed by the processor, cause:
prompting the speaker for the audio signal containing the command.

### Claim 14 (depends on 13)

The non-transitory, computer-readable medium of claim 13, wherein the prompting the speaker comprises prompting using a synthesized speech.

### Claim 15

A system in a computing device for recognizing speech, comprising:
a processor; and
a memory operatively connect to the processor and storing a set of instructions that, when executed by the processor, cause:
separating, using a computer-based neural network model, a speaker's speech from an audio signal based on the speaker's audio profile, the audio signal including the speaker's speech and audio signals associated with speech from other speakers or environmental audio signals;
determining a command from the speech;
determining, from the audio signal, a first score reflecting a percentage of confidence in determining the command based on a frequency of using the command by the speaker;
determining, from the audio signal, a second score reflecting a percentage of importance of the command;
determining whether the first score is above a first threshold value and the second score is below a second threshold value; and
causing the command to be executed at the computing device in response to determining that the first score is above the first threshold value and the second score is below the second threshold value.

### Claim 16 (depends on 15)

The system of claim 15, wherein the memory stores further instructions that, when executed by the processor, cause:
reducing background noise prior to the separating the speaker's speech from the audio signal based on the speaker's audio profile.

### Claim 17 (depends on 15)

The system of claim 15, wherein the separating the speaker's speech from the audio signal comprises separating the speaker's speech based on one of an audio waveform, a cadence of the speech, a pitch of the speech, loudness of the speech, or vocabulary of the speech.

### Claim 18 (depends on 15)

The system of claim 15, wherein the memory stores further instructions that, when executed by the processor, cause:
receiving video data of the speaker producing the audio signal; and
wherein the separating the speaker's speech comprises separating the speaker's speech based on a correlation of the audio signal and the video data.

### Claim 19 (depends on 15)

The system of claim 15, wherein the memory stores further instructions that, when executed by the processor, cause:
receiving vibration data of the speaker producing the audio signal; and
wherein the separating the speaker's speech comprises separating the speaker's speech based on a correlation of the audio signal and the vibration data.

### Claim 20 (depends on 15)

The system of claim 15, wherein the memory stores further instructions that, when executed by the processor, cause:
prompting the speaker for the audio signal containing the command.