# US 12,119,012 B2 · Method and apparatus for voice recognition in mixed audio based on pitch features using network models, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,119,012 B2
- Document ID: US12119012B2
- Status: Granted
- Kind: B2
- Application number: 17/353,636
- Filed: 2021-06-21
- Published: 2022-08-25
- Granted: 2024-10-15
- Earliest claimed: 2021-02-25
- Assignees: Beijing Xiaomi Pinecone Electronics Co., Ltd.
- Applicants: Beijing Xiaomi Pinecone Electronics Co., Ltd.
- Inventors: Na Xu; Yongtao Jia; Linzhang Wang
- Web page: https://patentgrep.com/patent/US12119012B2

## CPC

- G10L 21/007: characterised by the process used
- G10L 21/028: using properties of sound source
- G10L 21/0272: Voice signal separating
- G10L 25/30: using neural networks
- G10L 25/51: for comparison or discrimination
- G10L 17/02: Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction
- G10L 17/18: Artificial neural networks; Connectionist approaches
- G10L 25/90: Pitch determination of speech signals
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

The present disclosure relates to a method and an apparatus for audio processing and a storage medium. The method includes: obtaining an audio mixing feature of a target object, in which the audio mixing feature at least includes: a voiceprint feature and a pitch feature of the target object; and determining a target audio matching with the target object in the mixed audio according to the audio mixing feature.

## Claims

### Claim 1

A method for voice recognition in a mixed audio, comprising:
obtaining an audio signal of a target object;
extracting a voiceprint feature of the target object according to a short-time Fourier transform amplitude spectrum of the audio signal;
inputting the voiceprint feature into a predetermined pitch extraction network model to obtain a pitch feature, wherein the pitch feature refers to a lowest audio frequency that the target object can speak;
determining an audio mixing feature of the target object at least comprising the voiceprint feature and the pitch feature, wherein the audio mixing feature is used as a separation parameter in a voice separation process;
obtaining each audio feature of the mixed audio on each audio frame;
inputting each audio feature on each audio frame and the audio mixing feature into each sub-module of a separation network model to obtain an output result of each sub-module; and
obtaining a target audio matching with the target object in the mixed audio according to an overall output result of connecting the output results of all the sub-modules in series;
wherein the method further comprises: obtaining the mixed audio during a voice call, wherein the mixed audio comprises the target audio and noises, and outputting an enhanced target audio by performing noise reduction and amplification on the target audio after the voice separation process.

### Claim 2 (depends on 1)

The method of claim 1, wherein extracting the voiceprint feature of the target object comprises:
inputting the Short-Time Fourier Transform amplitude spectrum of the audio signal into a predetermined voiceprint extraction network model to obtain the voiceprint feature of the target object.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
obtaining registration information carrying a voice of the target object; and
obtaining the audio mixing feature according to the registration information.

### Claim 4 (depends on 1)

The method of claim 1, wherein the mixed audio at least comprises voice audios of two different objects, and the voice audios of the two different objects comprise voice audios of different objects with a same gender.

### Claim 5

An apparatus for voice recognition in a mixed audio, comprising:
one or more processors;
a memory storing instructions executable by the one or more processors;
wherein the one or more processors are configured to:
obtain an audio signal of a target object;
extract a voiceprint feature of the target object according to a short-time Fourier transform amplitude spectrum of the audio signal;
input the voiceprint feature into a predetermined pitch extraction network model to obtain a pitch feature, wherein the pitch feature refers to a lowest audio frequency that the target object can speak;
determine an audio mixing feature of the target object at least comprising the voiceprint feature and the pitch feature, wherein the audio mixing feature is used as a separation parameter in a voice separation process;
obtain each audio feature of the mixed audio on each audio frame;
input each audio feature on each audio frame and the audio mixing feature into each sub-module of a separation network model to obtain an output result of each sub-module; and
obtain a target audio matching with the target object in the mixed audio according to an overall output result of connecting the output results of all the sub-modules in series;
wherein the one or more processors are configured to:
obtain the mixed audio during a voice call, wherein the mixed audio comprises the target audio and noises, and
output an enhanced target audio by performing noise reduction and amplification on the target audio after the voice separation process.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the one or more processors are configured to:
input the short-time Fourier transform amplitude spectrum of the audio signal into a predetermined voiceprint extraction network model to obtain the voiceprint feature of the target object.

### Claim 7 (depends on 5)

The apparatus of claim 5, wherein the one or more processors are configured to:
obtain registration information carrying a voice of the target object; and
obtain the audio mixing feature according to the registration information.

### Claim 8 (depends on 5)

The apparatus of claim 5, wherein the mixed audio at least comprises voice audios of two different objects, and the voice audios of the two different objects comprise voice audios of different objects with a same gender.

### Claim 9

A non-transitory computer-readable storage medium having computer-executable instructions stored thereon, wherein when the computer-executable instructions are executed by a processor, the processor is caused to implement a method for voice recognition in a mixed audio, and the method comprises:
obtaining an audio signal of a target object;
extracting a voiceprint feature of the target object according to a short-time Fourier transform amplitude spectrum of the audio signal;
inputting the voiceprint feature into a predetermined pitch extraction network model to obtain a pitch feature, wherein the pitch feature refers to a lowest audio frequency that the target object can speak;
determining an audio mixing feature of the target object at least comprising the voiceprint feature and the pitch feature, wherein the audio mixing feature is used as a separation parameter in a voice separation process;
obtaining each audio feature of the mixed audio on each audio frame;
inputting each audio feature on each audio frame and the audio mixing feature into each sub-module of a separation network model to obtain an output result of each sub-module; and
obtaining a target audio matching with the target object in the mixed audio according to an overall output result of connecting the output results of all the sub-modules in series;
wherein the method further comprises: obtaining the mixed audio during a voice call wherein the mixed audio comprises the target audio and noises, and outputting an enhanced target audio by performing noise reduction and amplification on the target audio after the voice separation process.