Image Diffusion Framework for Text-Guided Video Editing
AssigneeUniversity of Central Florida Research Foundation, Inc.
InventorsNazmul Karim, Nazanin Rahnavard, Umar Khalid, Chen Chen
Life of this patent
Estimating the term…
Loading patent…
AssigneeUniversity of Central Florida Research Foundation, Inc.
InventorsNazmul Karim, Nazanin Rahnavard, Umar Khalid, Chen Chen
Estimating the term…
The invention provides a method for adapting a text-to-image (T2I) diffusion model for video editing by using spectral decomposition to achieve controlled spectral shifts in the model's weights. This adaptation involves maintaining constant singular vectors while selectively adjusting singular values in response to a text prompt. A spectral shift regularizer constrains adjustments, particularly limiting changes to larger singular values to ensure minimal deviation from the original model's structure. This approach allows efficient, prompt-driven video editing by modifying specific elements according to the prompt while preserving the original video context. By focusing on selective spectral adjustments, the method reduces adaptation time and computational demands, making it suitable for real-time and resource-sensitive applications, such as dynamic video editing for streaming services.
A method for adapting a text-to-image (T2I) diffusion model to edit video content based on a text prompt, the method comprising:
A method for adapting a text-to-image (T2I) diffusion model for text-driven video editing, the method comprising:
A method for zero-shot video generation from a text prompt using a pre-trained text-to-image (T2I) diffusion model, the method comprising:
Dependencies are inferred from the claim text.