Generatively pretrained models for speech restoration and extraction
AssigneeNVIDIA Corporation
InventorsPin-Jui Ku, Ante Jukic
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques related to a generative signal foundation model for temporal signal extraction and restoration. In at least one embodiment, a temporal signal is mapped into a latent space using an invertible transform, sampled and processed in the latent space, and the processed samples are inverse transformed to produce a restored or an output signal comprising an extracted target temporal signal or a signal with the target temporal signal removed. In at least one embodiment, the generative signal foundation model is pretrained using unlabeled training data and is then finetuned for a specific task.
Claims22
A computer-implemented method, comprising:
- masking an input temporal signal using a masking value to at least partially remove at least a portion of the input temporal signal to produce a masked temporal signal;
- applying one or more parameters to the masked temporal signal using a transformer-based neural network model to produce a vector field of coefficients in an invertible domain;
- and
- synthesizing an output temporal signal corresponding to a restored version of the input temporal signal by applying an inverse transform associated with the invertible domain.
A system for comprising:
- a memory that stores an input speech signal;
- and
- one or more processors coupled to the memory to perform operations including:
- masking the input speech signal using a masking value to at least partially remove at least a portion of the input speech signal to produce a masked speech signal;
- applying one or more parameters to the masked speech signal by a transformer-based neural network model to produce a vector field of coefficients in an invertible domain;
- and
- synthesizing an output speech signal corresponding to a restored version of the input speech signal by applying an inverse transform associated with the invertible domain.
A processor comprising:
- one or more processing units to synthesize an output temporal speech signal corresponding to a restored version of an input temporal speech signal by applying an inverse transform comprising at least a vector field of coefficients in an invertible domain to the input temporal speech signal, wherein the vector field of coefficients of the invertible domain is produced by using a transformer-based neural network to apply one or more parameters to the input temporal speech signal after masking at least a portion of the input temporal speech signal.
Dependencies are inferred from the claim text.