System and method for replicating background acoustic properties using neural networks
AssigneeMicrosoft Technology Licensing, LLC
InventorsDushyant SHARMA, James Wellford FOSBURGH, Patrick Aubrey NAYLOR
Life of this patent
Estimating the term…
Abstract
A method, computer program product, and computing system for estimating noise spectrum from a target audio signal segment. An acoustic neural embedding is generated from the target audio signal segment. An augmented audio signal segment is generated with background acoustic properties of the target audio signal segment by processing an input audio signal segment with the noise spectrum and the acoustic neural embedding using a neural network.
Claims20
A method comprising:
- receiving a target audio signal segment recorded in a target acoustic environment, wherein a speech processing system is deployed in the target acoustic environment;
- receiving an input audio signal segment generated by a text-to-speech (TTS) system, wherein background acoustic properties of the input audio signal segment mismatch background acoustic properties of the target audio signal segment;
- generating an acoustic neural embedding from the target audio signal segment;
- generating an augmented audio signal segment with background acoustic properties matching the background acoustic properties of the target audio signal segment by processing the input audio signal segment to add noise and reverberation in accordance the acoustic neural embedding;
- and
- training the speech processing system based on training data that includes the augmented audio signal segment.
A system comprising:
- at least one processor;
- and
- memory storing programming instructions for execution by the at least one processor, wherein the programming instructions, upon execution by the at least on processor, causes the system to perform the following operations:
- receiving a target audio signal segment recorded in a target acoustic environment, wherein a speech processing system is deployed in the target acoustic environment;
- receiving an input audio signal segment generated by a text-to-speech (TTS) system, wherein background acoustic properties of the input audio signal segment mismatch background acoustic properties of the target audio signal segment;
- generating an acoustic neural embedding from the target audio signal segment;
- generating an augmented audio signal segment with background acoustic properties matching the background acoustic properties of the target audio signal segment by processing the input audio signal segment to add noise and reverberation in accordance the acoustic neural embedding;
- and
- training the speech processing system based on training data that includes the augmented audio signal segment.
A computer program product residing on a non-transitory computer readable medium having programming instructions stored thereon which, when executed by at least one processor of a system, cause the system to perform the following operations:
- receiving a target audio signal segment recorded in a target acoustic environment, wherein a speech processing system is deployed in the target acoustic environment;
- receiving an input audio signal segment generated by a text-to-speech (TTS) system, wherein background acoustic properties of the input audio signal segment mismatch background acoustic properties of the target audio signal segment;
- generating an acoustic neural embedding from the target audio signal segment;
- generating an augmented audio signal segment with background acoustic properties matching the background acoustic properties of the target audio signal segment by processing the input audio signal segment to add noise and reverberation in accordance the acoustic neural embedding;
- and
- training the speech processing system based on training data that includes the augmented audio signal segment.
Dependencies are inferred from the claim text.