Method, system, and computer-readable medium for stylizing video frames
AssigneeGuangdong Oppo Mobile Telecommunications Corp., Ltd.
InventorJenhao HSIAO
Life of this patent
Estimating the term…
Abstract
In an embodiment, a method includes receiving first and second images of a video sequence, wherein the first and second images are consecutive image frames; applying a style network model to the first and second images to generate first and second stylized images in a style of a style image, respectively; applying a loss network model to the first and second images, the first and second stylized images, and the style image to generate a loss function; determining a set of weights for the style network model based on the generated loss function; and stylizing the video frames using the style network model. The method can mitigate flicker artifacts between the stylized consecutive frames.
Claims20
A method for stylizing video frames, comprising:
- receiving a first image and a second image of a video sequence, wherein the first image and the second image are consecutive image frames;
- applying a style network model associated with a style image to the first image and the second image to generate a first stylized image and a second stylized image in a style of the style image, respectively;
- applying a loss network model to the first image, the second image, the first stylized image, the second stylized image, and the style image to generate a loss function;
- determining a set of weights for the style network model based on the generated loss function;
- and
- stylizing, by at least one processor, the video frames by applying the style network model with the determined set of weights to the video frames.
A system for stylizing video frames, comprising:
- at least one memory configured to store program instructions;
- at least one processor configured to execute the program instructions, which cause the at least one processor to perform steps comprising:
- receiving a first image and a second image of a video sequence, wherein the first image and the second image are consecutive image frames;
- applying a style network model associated with a style image to the first image and the second image to generate a first stylized image and a second stylized image in a style of the style image, respectively;
- applying a loss network model to the first image, the second image, the first stylized image, the second stylized image, and the style image to generate a loss function;
- determining a set of weights for the style network model based on the generated loss function;
- and
- stylizing the video frames by applying the style network model with the determined set of weights to the video frames.
A non-transitory computer-readable medium with program instructions stored thereon, that when executed by at least one processor, cause the at least one processor to perform steps comprising:
- receiving a first image and a second image of a video sequence, wherein the first image and the second image are consecutive image frames;
- applying a style network model associated with a style image to the first image and the second image to generate a first stylized image and a second stylized image in a style of the style image, respectively;
- applying a loss network model to the first image, the second image, the first stylized image, the second stylized image, and the style image to generate a loss function;
- determining a set of weights for the style network model based on the generated loss function;
- and
- stylizing the video frames by applying the style network model with the determined set of weights to the video frames.
Dependencies are inferred from the claim text.