Diffusion inference-time t-optimization for music generation
AssigneeAdobe Inc.
InventorsZachary NOVACK, Nicholas J. BRYAN
Life of this patent
Estimating the term…
Abstract
Embodiments are disclosed for music generation. The method may include receiving a text input describing music to be generated by a neural network and obtaining an initial noise laten. The neural network generates a music spectrogram based on the initial noise latent and the text input. One or more features are extracted from the music spectrogram. A loss is determined based on the one or more features from the music spectrogram and one or more target features of a target output. An optimized noise latent is obtained based on the loss and a new music spectrogram is generated using the optimized noise latent.
Claims20
A method comprising:
- receiving a text input describing music to be generated by a neural network;
- obtaining an initial noise latent;
- generating, by the neural network, a music spectrogram based on the initial noise latent and the text input;
- extracting one or more features from the music spectrogram;
- determining a loss based on the one or more features from the music spectrogram and one or more target features of a target output;
- obtaining an optimized noise latent based on the loss;
- and
- generating a new music spectrogram using the optimized noise latent.
A non-transitory computer-readable medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
- receiving a text input describing music to be generated by a neural network;
- obtaining an initial noise latent;
- generating, by the neural network, a music spectrogram based on the initial noise latent and the text input;
- extracting one or more features from the music spectrogram;
- determining a loss based on the one or more features from the music spectrogram and one or more target features of a target output;
- obtaining an optimized noise latent based on the loss;
- and
- generating a new music spectrogram using the optimized noise latent.
A system comprising:
- a memory component;
- and
- a processing device coupled to the memory component, the processing device to perform operations comprising:
- receiving a request to modify a reference music data;
- determining an overlap region associated with an end of the reference music data;
- generating, by a neural network, music data based on an initial noise latent;
- extracting one or more features from the music data and the overlap region;
- determining a loss based on the one or more features from the music data and the overlap region;
- obtaining an optimized noise latent based on the loss;
- generating new music data using the optimized noise latent;
- and
- stitching together the reference music data and the new music data.
Dependencies are inferred from the claim text.