High fidelity audio super resolution
AssigneesAdobe Inc., The Trustees of Princeton University
InventorsZeyu Jin, Jiaqi Su, Adam Finkelstein
Life of this patent
Estimating the term…
Abstract
Embodiments are disclosed for generating full-band audio from narrowband audio using a GAN-based audio super resolution model. A method of generating full-band audio may include receiving narrow-band input audio data, upsampling the narrow-band input audio data to generate upsampled audio data, providing the upsampled audio data to an audio super resolution model, the audio super resolution model trained to perform bandwidth expansion from narrow-band to wide-band, and returning wide-band output audio data corresponding to the narrow-band input audio data.
Claims17
A computer-implemented method comprising:
- receiving narrow-band input audio data;
- dividing the narrow-band input audio data into a plurality of batches;
- upsampling each batch of the narrow-band input audio data to generate upsampled audio batch data;
- providing the upsampled audio batch data to an audio super resolution model, wherein the audio super resolution model is a generator neural network trained as part of a generative adversarial network (GAN) to generate wide-band audio data from narrow-band audio data;
- generating, by the audio super resolution model, wide-band audio batch data corresponding to each batch of the plurality of batches, the wide-band audio batch data including high frequency data not included in the narrow-band input audio data;
- combining the wide-band audio batch data corresponding to each batch of the plurality of batches to form wide-band output audio data;
- and
- returning the wide-band output audio data corresponding to the narrow-band input audio data.
A non-transitory computer-readable storage medium including instructions stored thereon which, when executed by at least one processor, cause the at least one processor to:
- receive narrow-band input audio data;
- divide the narrow-band input audio data into a plurality of batches;
- upsample each batch of the narrow-band input audio data to generate upsampled audio batch data;
- provide the upsampled audio batch data to an audio super resolution model, wherein the audio super resolution model is a generator neural network trained as part of a generative adversarial network (GAN) to generate wide-band audio data from narrow-band audio data;
- generate, by the audio super resolution model, wide-band audio batch data corresponding to each batch of the plurality of batches, the wide-band audio batch data including high frequency data not included in the narrow-band input audio data;
- combine the wide-band audio batch data corresponding to each batch of the plurality of batches to form wide-band output audio data;
- and
- return the wide-band output audio data corresponding to the narrow-band input audio data.
A system comprising:
- a processor;
- and
- a memory including instructions stored thereon which, when executed by the processor, cause the system to:
- receive narrow-band input audio data;
- divide the narrow-band input audio data into a plurality of batches;
- upsample each batch of the narrow-band input audio data to generate upsampled audio batch data;
- provide the upsampled audio batch data to an audio super resolution model, wherein the audio super resolution model is a generator neural network trained as part of a generative adversarial network (GAN) to generate wide-band audio data from narrow-band audio data;
- generate, by the audio super resolution model, wide-band audio batch data corresponding to each batch of the plurality of batches, the wide-band audio batch data including high frequency data not included in the narrow-band input audio data;
- combine the wide-band audio batch data corresponding to each batch of the plurality of batches to form wide-band output audio data;
- and
- return the wide-band output audio data corresponding to the narrow-band input audio data.
Dependencies are inferred from the claim text.