# US 12,112,767 B2 · Acoustic data augmentation with mixed normalization factors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,767 B2
- Document ID: US12112767B2
- Status: Granted
- Kind: B2
- Application number: 17/326,463
- Filed: 2021-05-21
- Published: 2022-11-24
- Granted: 2024-10-08
- Earliest claimed: 2021-05-21
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Toru Nagano; Takashi Fukuda; Masayuki Suzuki
- Web page: https://patentgrep.com/patent/US12112767B2

## CPC

- G10L 15/063: {Training}
- G10L 21/00: Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility (G10L19/00 takes precedence)
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 20/00: Machine learning
- G10L 19/02: using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L 25/27: characterised by the analysis technique
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method, computer system, and a computer program product for audio data augmentation are provided. Sets of audio data from different sources may be obtained. A respective normalization factor for at least two sources of the different sources may be calculated. The normalization factors from the at least two sources may be mixed to determine a mixed normalization factor. A first set of the sets may be normalized by using the mixed normalization factor and to obtain training data for training an acoustic model.

## Claims

### Claim 1

A method for acoustic data augmentation, the method comprising:
obtaining sets of audio data from different sources, wherein each of the respective sets comprises audio values with respect to time values over a respective time period;
calculating a respective normalization factor for at least four sets of the sets of audio data, wherein the audio values of the at least four sets are for a same type of audio value;
calculating a mixed normalization factor by using the at least four calculated normalization factors, wherein the mixed normalization factor is located at any point within a quadrilateral that includes the at least four calculated normalization factors;
normalizing at least two sets of the at least four sets by using the mixed normalization factor, wherein the normalized at least two sets together constitute training data; and
training an acoustic model by using the training data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the different sources are different speakers.

### Claim 3 (depends on 1)

The method of claim 1, wherein the audio data is selected from a group consisting of raw speech data, log Mel-filtered spectra generated from raw speech data, and Mel-Frequency Cepstrum Coefficients transformed from raw speech data.

### Claim 4 (depends on 1)

The method of claim 1, wherein the calculating the normalization factor for a first set of the at least four sets is performed by using z-score normalization; and
wherein the normalization factor for the first set comprises a respective mean value and a respective standard deviation for the z-score normalization.

### Claim 5 (depends on 1)

The method of claim 1, wherein a first set of the at least four sets of audio data is from a first source of the different sources;
wherein a second set of the at least four sets of audio data is from a second source of the different sources; and
wherein a third set of the at least four sets of audio data is from a third source of the different sources.

### Claim 6 (depends on 1)

The method of claim 1, further comprising decoding audio data by using the trained acoustic model to generate sound identification information.

### Claim 7 (depends on 1)

The method of claim 1, wherein the data sets are recorded under different acoustic environments.

### Claim 8 (depends on 1)

The method of claim 1, wherein the calculating the mixed normalization factor comprises applying weights to the at least four calculated normalization factors, the weights in total adding to 1.

### Claim 9 (depends on 1)

The method of claim 1, wherein the data sets are grouped via unsupervised clustering.

### Claim 10

A computer system for acoustic data augmentation, the computer system comprising:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored on at least one of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing a method comprising:
obtaining sets of audio data from different sources, wherein each of the respective sets comprises audio values with respect to time values over a respective time period;
calculating a respective normalization factor for at least four sets of the sets of audio data, wherein the audio values of the at least four sets are for a same type of audio value;
calculating a mixed normalization factor by using the at least four calculated normalization factors, wherein the mixed normalization factor is located at any point within a quadrilateral that includes the at least four calculated normalization factors;
normalizing at least two sets of the at least four sets by using the mixed normalization factor, wherein the normalized at least two sets together constitute training data; and
training an acoustic model by using the training data.

### Claim 11 (depends on 10)

The computer system of claim 10, wherein the different sources are different speakers.

### Claim 12 (depends on 10)

The computer system of claim 10, wherein the audio data is selected from a group consisting of raw speech data, log Mel-filtered spectra generated from raw speech data, and Mel-Frequency Cepstrum Coefficients transformed from raw speech data.

### Claim 13 (depends on 10)

The computer system of claim 10, wherein the calculating the normalization factor for a first set of the at least four sets is performed by using z-score normalization; and
wherein the normalization factor for the first set comprises a respective mean value and a respective standard deviation for the z-score normalization.

### Claim 14 (depends on 10)

The computer system of claim 10, wherein the data sets are recorded under different acoustic environments.

### Claim 15

A computer program product for acoustic data augmentation, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the program instructions are executable by a computer system to cause the computer system to perform a method comprising:
obtaining sets of audio data from different sources, wherein each of the respective sets comprises audio values with respect to time values over a respective time period;
calculating a respective normalization factor for at least four sets of the sets of audio data, wherein the audio values of the at least four sets are for a same type of audio value;
calculating a mixed normalization factor by using the at least four calculated normalization factors, wherein the mixed normalization factor is located at any point within a quadrilateral that includes the at least four calculated normalization factors;
normalizing at least two sets of the at least four sets by using the mixed normalization factor, wherein the normalized at least two sets together constitute training data; and
training an acoustic model by using the training data.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the different sources are different speakers.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein the audio data is selected from a group consisting of raw speech data, log Mel-filtered spectra generated from raw speech data, and Mel-Frequency Cepstrum Coefficients transformed from raw speech data.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein the calculating the normalization factor for a first set of the at least four sets is performed by using z-score normalization; and
wherein the normalization factor for the first set comprises a respective mean value and a respective standard deviation for the z-score normalization.

### Claim 19 (depends on 15)

The computer program product of claim 15, wherein the calculating the mixed normalization factor comprises applying weights to the at least four calculated normalization factors, the weights in total adding to 1.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein the data sets are recorded under different acoustic environments.