# US 2023/0410794 A1 · Audio recognition method, method of training audio recognition model, and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410794 A1
- Document ID: US20230410794A1
- Status: Published application
- Kind: A1
- Application number: 18/237,976
- Filed: 2023-08-25
- Published: 2023-12-21
- Earliest claimed: 2022-09-02
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Xiaoyin FU; Mingshun YANG; Qiguang ZANG; Zhijie CHEN; Yangkai XU; Guibin WANG; Lei JIA
- Web page: https://patentgrep.com/patent/US20230410794A1

## CPC

- G10L 15/063: {Training}
- G10L 15/08: Speech classification or search
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 15/16: using artificial neural networks
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An audio recognition method, a method of training an audio recognition model, and an electronic device are provided, which relate to fields of artificial intelligence, speech recognition, deep learning and natural language processing technologies. The audio recognition method includes: truncating an audio feature of target audio data to obtain at least one first audio sequence feature corresponding to a predetermined duration; obtaining, according to a peak information of the audio feature, a peak sub-information corresponding to the first audio sequence feature; performing at least one decoding operation on the first audio sequence feature to obtain a recognition result for the first audio sequence feature, a number of times the decoding operation is performed being identical to a number of peaks corresponding to the first audio sequence feature; obtaining target text data for the target audio data according to the recognition result for the at least one first audio sequence feature.

## Claims

### Claim 1

An audio recognition method, comprising:
truncating an audio feature of target audio data to obtain at least one first audio sequence feature, wherein a duration corresponding to the at least one first audio sequence feature is a predetermined duration;
obtaining, according to a peak information of the audio feature, a peak sub-information corresponding to the first audio sequence feature, wherein the peak sub-information indicates a peak corresponding to the first audio sequence feature;
performing at least one decoding operation on the first audio sequence feature to obtain a recognition result for the first audio sequence feature, wherein a number of times the decoding operation is performed is identical to a number of peaks corresponding to the first audio sequence feature; and
obtaining target text data for the target audio data according to the recognition result for the at least one first audio sequence feature.

### Claim 2 (depends on 1)

The method according to claim 1, wherein a number of first audio sequence features is K, a recognition result for a kth first audio sequence feature among K first audio sequence features comprises I recognition sub-results, the kth first audio sequence feature corresponds to I peaks, wherein I is an integer greater than or equal to 1, k is an integer greater than or equal to 1 and less than or equal to K, and K is an integer greater than 1.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the performing at least one decoding operation on the first audio sequence feature comprises:
performing an ith decoding operation on the kth first audio sequence feature according to an (i−1)th decoding parameter information of the kth first audio sequence feature, so as to obtain an ith decoding parameter information of the kth first audio sequence feature and an ith recognition sub-result for the kth first audio sequence feature, wherein i is an integer greater than 1 and less than or equal to I.

### Claim 4 (depends on 2)

The method according to claim 2, wherein the performing at least one decoding operation on the first audio sequence feature comprises:
performing a 1st decoding operation on the kth first audio sequence feature according to an initial decoding parameter information of the kth first audio sequence feature, so as to obtain a 1st decoding parameter information of the kth first audio sequence feature and a 1st recognition sub-result for the kth first audio sequence feature.

### Claim 5 (depends on 3)

The method according to claim 3, wherein I is an integer greater than 1, and the performing an ith decoding operation on the kth first audio sequence feature comprises:
performing an Ith decoding operation on the kth first audio sequence feature according to an (I−1)th decoding parameter information of the kth first audio sequence feature, so as to obtain an Ith decoding parameter information of the kth first audio sequence feature and an Ith recognition sub-result for the kth first audio sequence feature.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the performing an Ith decoding operation on the kth first audio sequence feature further comprises:
in a case that k is less than K, using the Ith decoding parameter information of the kth first audio sequence feature as an initial decoding parameter information of a (k+1)th first audio sequence feature.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the performing at least one decoding operation on the first audio sequence feature comprises:
performing, in response to a determination that the first audio sequence feature meets a recognition start condition, the at least one decoding operation on the first audio sequence feature according to a first predetermined decoding parameter information, so as to obtain an original decoding parameter information and the recognition result for the first audio sequence feature.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the obtaining target text data for the target audio data according to the recognition result for the at least one first audio sequence feature comprises:
performing, in response to a second audio sequence feature being truncated from the audio feature, at least one decoding operation on the second audio sequence feature according to a second predetermined decoding parameter information, so as to obtain a recognition result for the second audio sequence feature, wherein the second audio sequence feature meets a recognition end condition; and
obtaining the target text data according to the recognition result for the at least one first audio sequence feature and the recognition result for the second audio sequence feature.

### Claim 9 (depends on 2)

The method according to claim 2, wherein the performing at least one decoding operation on the first audio sequence feature comprises:
encoding the kth first audio sequence feature to obtain a kth initial audio sequence encoding feature;
obtaining a kth target audio sequence encoding feature according to the kth initial audio sequence encoding feature; and
performing at least one decoding operation on the kth target audio sequence encoding feature to obtain the recognition result for the first audio sequence feature.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the obtaining a kth target audio sequence encoding feature according to the kth initial audio sequence encoding feature comprises:
obtaining the kth target audio sequence encoding feature according to the kth initial audio sequence encoding feature and a historical feature related to the kth first audio sequence feature.

### Claim 11 (depends on 9)

The method according to claim 9, wherein the performing at least one decoding operation on the first audio sequence feature comprises:
obtaining a 1st historical sub-feature of the kth first audio sequence feature according to the kth initial audio sequence encoding feature and a 1st recognition sub-result for the kth first audio sequence feature;
obtaining an ith historical sub-feature of the kth first audio sequence feature according to the kth initial audio sequence encoding feature and an ith recognition sub-result for the kth first audio sequence feature, wherein i is an integer greater than 1 and less than or equal to I; and
fusing I historical sub-features of the kth first audio sequence feature and the historical feature related to the kth first audio sequence feature to obtain a historical feature related to a (k+1)th first audio sequence feature.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the obtaining a peak sub-information corresponding to the first audio sequence feature according to a peak information of the audio feature comprises:
obtaining the peak information of the audio feature according to the audio feature, wherein the peak information indicates a peak corresponding to the audio feature, and the peak corresponds to a predetermined value; and
obtaining the peak sub-information corresponding to the first audio sequence feature according to the peak information and the first audio sequence feature.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the predetermined value indicates that the peak corresponds to a semantic unit, and predetermined values corresponding to different peaks are identical to each other.

### Claim 14 (depends on 12)

The method according to claim 12, wherein the audio feature comprises N audio sub-features, the audio sub-feature corresponds to a time instant, and N is an integer greater than or equal to 1, and
wherein the obtaining the peak information of the audio feature according to the audio feature comprises:
performing a time masking on the audio feature to obtain a time-masked feature, wherein the time-masked feature corresponds to a 1st audio sub-feature to an nth audio sub-feature, and n is an integer greater than 1 and less than N; and
obtaining, according to the time-masked feature, peak information corresponding to n time instants, and
wherein the obtaining peak information corresponding to n time instants according to the time-masked feature comprises:
performing a convolution on the time-masked feature to obtain a convoluted time-masked feature; and
obtaining the peak information corresponding to the n time instants according to the convoluted time-masked feature.

### Claim 15 (depends on 1)

The method according to claim 1, wherein the truncating an audio feature of target audio data comprises:
performing a convolution on the audio feature to obtain a first audio feature; and
truncating the first audio feature, and
wherein the truncating the first audio feature comprises:
truncating the first audio feature in response to a determination that a duration corresponding to the first audio feature meets a predetermined duration condition.

### Claim 16 (depends on 1)

The method according to claim 1, wherein the obtaining a peak sub-information corresponding to the first audio sequence feature according to a peak information of the audio feature comprises:
performing a convolution on the audio feature to obtain a second audio feature; and
obtaining the peak sub-information corresponding to the first audio sequence feature according to a peak information of the second audio feature.

### Claim 17 (depends on 1)

The method according to claim 1, wherein a number of target audio data is multiple, and a number of audio features is multiple, and
wherein the performing at least one decoding operation on the first audio sequence feature comprises:
performing the at least one decoding operation in parallel on the first audio sequence features respectively obtained from the plurality of audio features.

### Claim 18 (depends on 2)

The method according to claim 2, wherein the first audio sequence feature comprises J audio sequence sub-features, and J is an integer greater than 1, and
wherein the kth first audio sequence feature comprises a (J−H)th audio sequence sub-feature of a (k−1)th first audio sequence feature, and H is an integer greater than or equal to 0.

### Claim 19

A method of training an audio recognition model, wherein the audio recognition model comprises a recognition sub-model, the method comprising:
truncating an audio feature of sample audio data by using the recognition sub-model, so as to obtain at least one first audio sequence feature, wherein a duration corresponding to the at least one first audio sequence feature is a predetermined duration;
obtaining, according to a sample peak information of the audio feature, a sample peak sub-information corresponding to the first audio sequence feature, wherein the sample peak sub-information indicates a sample peak corresponding to the first audio sequence feature;
performing at least one decoding operation on the first audio sequence feature by using the recognition sub-model, so as to obtain a recognition result for the first audio sequence feature, wherein a number of times the decoding operation is performed is identical to a number of sample peaks corresponding to the first audio sequence feature;
obtaining sample text data for the sample audio data according to the recognition result for the at least one first audio sequence feature;
determining a recognition loss value according to the sample text data and a recognition sub-label of the sample audio data; and
training the audio recognition model according to the recognition loss value.

### Claim 20

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions, when executed by the at least one processor, are configured to cause the at least one processor to at least:
truncate an audio feature of target audio data to obtain at least one first audio sequence feature, wherein a duration corresponding to the at least one first audio sequence feature is a predetermined duration;
obtain, according to a peak information of the audio feature, a peak sub-information corresponding to the first audio sequence feature, wherein the peak sub-information indicates a peak corresponding to the first audio sequence feature;
perform at least one decoding operation on the first audio sequence feature to obtain a recognition result for the first audio sequence feature, wherein a number of times the decoding operation is performed is identical to a number of peaks corresponding to the first audio sequence feature; and
obtain target text data for the target audio data according to the recognition result for the at least one first audio sequence feature.
