# US 2024/0404544 A1 · Audio distortion removal based on a set of reference audio samples > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0404544 A1 - Document ID: US20240404544A1 - Status: Published application - Kind: A1 - Application number: 18/329,401 - Filed: 2023-06-05 - Published: 2024-12-05 - Earliest claimed: 2023-06-05 - Assignees: Cypress Semiconductor Corporation - Applicants: Cypress Semiconductor Corporation - Inventors: Oleg KAPSHII; Igor KRAVETS; Andrii TSEMKO; Ashutosh PANDEY - Web page: https://patentgrep.com/patent/US20240404544A1 ## CPC - G10L 21/0208: Noise filtering - G10L 21/0264: characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques - G06N 20/00: Machine learning - G10L 15/1822: {Parsing for meaning understanding} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 25/06: the extracted parameters being correlation coefficients - G10L 25/60: for measuring the quality of voice signals - G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) - G10L 25/30: using neural networks - G10L 2015/088: {Word spotting} - G10L 2021/02082: {the noise being echo, reverberation of the speech} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method includes receiving, by a processing device, an audio sample of a distorted voice. The method further includes extracting a first set of characteristics from the audio sample. The method further includes selecting a second set of characteristics. The second set of characteristics is associated with one of a set of reference audio samples. The method further includes generating an undistorted audio sample. The undistorted audio sample is based on the distorted audio sample, the first set of characteristics, and the second set of characteristics. ## Claims ### Claim 1 A method, comprising: receiving, by a processing device, an audio sample of a distorted voice; extracting a first set of characteristics of the audio sample; selecting a second set of characteristics, wherein the second set of characteristics is associated with one of a first plurality of reference audio samples; and generating an undistorted audio sample based on the audio sample, the first set of characteristics, and the second set of characteristics. ### Claim 2 (depends on 1) The method of claim 1, wherein selecting the second set of characteristics comprises: providing the first set of characteristics to a trained machine learning model; and receiving, from the trained machine learning model, a selection of the second set of characteristics, wherein the trained machine learning model is configured to select the second set of characteristics from a plurality of sets of characteristics based on predicted performance of the generating of the undistorted audio sample based on the first set of characteristics and the second set of characteristics. ### Claim 3 (depends on 2) The method of claim 2, wherein the first set of characteristics comprises a plurality of auto regression coefficients associated with the audio sample, and wherein the trained machine learning model is further to generate a set of parameters for use in generating the plurality of auto regression coefficients, wherein the set of parameters is based on the audio sample. ### Claim 4 (depends on 2) The method of claim 2, wherein extracting the first set of characteristics comprises data smoothing, and wherein smoothing parameters are provided by the trained machine learning model. ### Claim 5 (depends on 2) The method of claim 2, wherein generating the trained machine learning model comprises: providing a first plurality of indications of sets of characteristics associated with reference audio samples to a machine learning model as training input; providing a first plurality of characteristics of distorted audio samples to the machine learning model as training input; generating a plurality of undistorted audio samples, wherein each of the plurality of undistorted audio samples is based on one of the first plurality of indications of sets of characteristics associated with reference audio samples and one of the first plurality of characteristics of distorted audio samples; determining of plurality of figures of merit indicative of a quality of the plurality of undistorted audio samples; providing the plurality of figures of merit to the machine learning model; and training the machine learning model based on the first plurality of indications of sets of characteristics associated with reference audio samples, the first plurality of characteristics of distorted audio samples, and the plurality of figures of merit. ### Claim 6 (depends on 5) The method of claim 5, wherein the generating a figure of merit of the plurality of figures of merit comprises comparing an undistorted audio sample to a corresponding distorted audio sample. ### Claim 7 (depends on 5) The method of claim 5, wherein generating the plurality of figures of merit comprises perceptual evaluation of speech quality (PESQ) operations. ### Claim 8 (depends on 1) The method of claim 1, wherein generating the undistorted audio sample comprises performing modulation transfer function dereverberation operations. ### Claim 9 (depends on 1) The method of claim 1, wherein generating the first plurality of reference audio samples comprises: receiving a second plurality of reference audio samples, wherein the first plurality of reference audio samples is a subset of the second plurality of reference audio samples; generating indications of performance of reference audio samples of the second plurality of reference audio samples in operations of generating undistorted audio samples based on audio samples of distorted voices; clustering reference audio samples of the second plurality of reference audio samples based on their performance; and selecting reference audio samples comprising the first plurality of reference audio samples based on the clustering. ### Claim 10 (depends on 1) The method of claim 1, wherein the audio sample comprises a frame of a plurality of frames, and wherein selecting the second set of characteristics is based on selections of sets of characteristics associated with other frames also associated with the distorted voice. ### Claim 11 (depends on 1) The method of claim 1, further comprising providing the undistorted audio sample to a speech recognition module configured to perform wake-word-detection operations or automated speech recognition operations; and obtaining output from the speech recognition module. ### Claim 12 A system, comprising memory and a processing device coupled to the memory, wherein the processing device is configured to: receive an audio sample of a distorted voice; extract a first set of characteristics of the audio sample; select a second set of characteristics, wherein the second set of characteristics is associated with one of a first plurality of reference audio samples; and generate an undistorted audio sample based on the audio sample, the first set of characteristics, and the second set of characteristics. ### Claim 13 (depends on 12) The system of claim 12, wherein selecting the second set of characteristics comprises: providing the first set of characteristics to a trained machine learning model; and receiving, from the trained machine learning model, a selection of the second set of characteristics from a plurality of sets of characteristics, wherein the trained machine learning model is configured to select the second set of characteristics based on predicted performance of the generating of the undistorted audio sample based on the first set of characteristics and the second set of characteristics. ### Claim 14 (depends on 13) The system of claim 13, wherein generating the trained machine learning model comprises: providing a first plurality of indications of sets of characteristics associated with reference audio samples to a machine learning model as training input; providing a first plurality of characteristics of distorted audio samples to the machine learning model as training input; generating a plurality of undistorted audio samples, wherein each of the plurality of undistorted audio samples is based on one of the first plurality of indications of sets of characteristics associated with reference audio samples and one of the first plurality of characteristics of distorted audio samples; determining of plurality of figures of merit indicative of a quality of the plurality of undistorted audio samples; providing the plurality of figures of merit to the machine learning model; and training the machine learning model based on the first plurality of indications of sets of characteristics associated with reference audio samples, the first plurality of characteristics of distorted audio samples, and the plurality of figures of merit. ### Claim 15 (depends on 12) The system of claim 12, wherein the first plurality of reference audio samples comprises a subset of a library of reference audio samples, and wherein generating the subset comprises: generating indications of performance of reference audio samples of the library of reference audio samples, wherein the indications of performance are based on a quality of undistorted audio samples generated based on audio samples of distorted voices; clustering reference audio samples of the library of reference audio samples based on the indications of performance; and selecting reference audio samples of the first plurality of reference audio samples based on the clustering. ### Claim 16 (depends on 12) The system of claim 12, wherein the processing device is further configured to: provide the undistorted audio sample to a speech parsing module configured to perform wake-word-detection operations or automated speech recognition operations; and obtain output from the speech parsing module based on the undistorted audio sample. ### Claim 17 A non-transitory machine-readable storage medium storing instruction which, when executed, cause a processing device to perform operations comprising: receiving an audio sample of a distorted voice; extracting a first set of characteristics of the audio sample; selecting a second set of characteristics, wherein the second set of characteristics is associated with one of a first plurality of reference audio samples; and generating an undistorted audio sample based on the audio sample, the first set of characteristics, and the second set of characteristics. ### Claim 18 (depends on 17) The non-transitory machine-readable storage medium of claim 17, wherein selecting the second set of characteristics comprises: providing the first set of characteristics to a trained machine learning model; and receiving, from the trained machine learning model, a selection of the second set of characteristics, wherein the trained machine learning model is configured to select the second set of characteristics from a plurality of sets of characteristics based on predicted performance of the generating of the undistorted audio sample based on the first set of characteristics and the second set of characteristics. ### Claim 19 (depends on 18) The non-transitory machine-readable storage medium of claim 18, wherein generating the trained machine learning model comprises: providing a first plurality of indications of sets of characteristics associated with reference audio samples to a machine learning model as training input; providing a first plurality of characteristics of distorted audio samples to the machine learning model as training input; generating a plurality of undistorted audio samples, wherein each of the plurality of undistorted audio samples is based on one of the first plurality of indications of sets of characteristics associated with reference audio samples and one of the first plurality of characteristics of distorted audio samples; determining of plurality of figures of merit indicative of a quality of the plurality of undistorted audio samples; providing the plurality of figures of merit to the machine learning model; and training the machine learning model based on the first plurality of indications of sets of characteristics associated with reference audio samples, the first plurality of characteristics of distorted audio samples, and the plurality of figures of merit. ### Claim 20 (depends on 17) The non-transitory machine-readable storage medium of claim 17, wherein the operations further comprise: providing the undistorted audio sample to a speech parsing module configured to perform wake-word-detection operations or automated speech recognition operations; and obtaining output, based on the undistorted audio sample, from the speech parsing module.