High resolution style transfer
AssigneeAdobe Inc.
InventorsZhifei Zhang, Zhe Lin, Zhaowen Wang
Life of this patent
Estimating the term…
Abstract
High resolution style transfer techniques and systems are described that overcome the challenges of transferring high resolution style features from one image to another image, and of the limited availability of training data to perform high resolution style transfer. In an example, a neural network is trained using high resolution style features which are extracted from a style image and are used in conjunction with an input image to apply the style features to the input image to generate a version of the input image transformed using the high resolution style features.
Claims20
In a digital medium high resolution style transfer environment, a method implemented by at least one computing device, the method comprising:
- generating, by the at least one computing device, a transferred image that includes content from an input image transformed by applying a first set of style attributes from a style image to the input image;
- extracting, by the at least one computing device, a second set of style attributes from an upscaled version of the style image, the second set of style attributes being of higher visual resolution than the first set of style attributes;
- generating, by the at least one computing device, a style image feature map that includes style features from the second set of style attributes, and a transferred image feature map that includes image features from an upscaled version of the transferred image;
- generating, by the at least one computing device, a swapped feature map by matching style features from the style image feature map with image features from the transferred image feature map;
- and
- generating, by the at least one computing device, a resolved image that includes higher resolution style attributes than the transferred image based on the swapped feature map.
In a digital medium high resolution style transfer environment, a system comprising:
- a processing system;
- and
- one or more computer-readable storage media storing instructions that are executable by the processing system to implement:
- a transfer module to generate a transferred image by applying a first set of style attributes from a style image to an input image to generate the transferred image to include content from the input image transformed using the first set of style attributes;
- a feature extractor module to extract a second set of style attributes from an upscaled version of the style image and generate a style image feature map that includes style features from the second set of style attributes, and to generate a transferred image feature map that includes image features from an upscaled version of the transferred image;
- a style swap module to generate a swapped feature map by matching style features from the style image feature map to image features from the transferred image feature map to generate the swapped feature map to include the matched style features;
- and
- a style resolution system to implement a resolution neural network to generate a resolved image that includes higher resolution style attributes than the transferred image and based on the swapped feature map.
In a digital medium high resolution style transfer environment, a system comprising:
- one or more processors;
- and
- one or more computer-readable storage media storing instructions that are executable by the one or more processing to perform operations including:
- generating, by the at least one computing device, a transferred image that includes content from an input image transformed by applying a first set of style attributes from a style image to the input image;
- extracting, by the at least one computing device, a second set of style attributes from an upscaled version of the style image, the second set of style attributes being of higher visual resolution than the first set of style attributes;
- generating, by the at least one computing device, a style image feature map that includes style features from the second set of style attributes, and a transferred image feature map that includes image features from an upscaled version of the transferred image;
- generating, by the at least one computing device, a swapped feature map by matching style features from the style image feature map with image features from the transferred image feature map;
- and
- generating, by the at least one computing device, a resolved image that includes higher resolution style attributes than the transferred image based on the swapped feature map.
Dependencies are inferred from the claim text.