Back to results
US 11,145,042 B2Granted
Using convolutional neural network style transfer to automate graphic design creation
AssigneePalo Alto Research Center Incorporated
InventorKalaivani Ramea Kubendran
Life of this patent
Estimating the term…
Abstract
A silhouette image and a style image are input to a convolutional neural network to produce respective content feature layer and pattern feature layers. A reference image is input into the convolutional neural network to determine reference feature layers. For each of a plurality of iterations, a combination loss of the convolutional neural network is minimized to obtain an output image comprising an abstraction of the style image within confines of the silhouette image. The combination loss includes a content loss based on the content feature layers and a style loss based on the pattern feature layers.
Claims18
A method comprising:
- inputting a silhouette image to a convolutional neural network to produce a content feature layer;
- receiving a user selection between two different style options, a first option utilizing all sublayers of lower layers of the convolutional neural network for pattern feature layers and performing more than five iterations, the second option utilizing only one sublayer of each lower layer of the convolutional neural network for the pattern feature layers and performing five or fewer iterations;
- inputting a style image to the convolutional neural network to produce the pattern feature layers according to the user selection;
- inputting a reference image into the convolutional neural network to determine reference feature layers, and for each of a plurality of iterations, minimizing a combination loss of the convolutional neural network to obtain an output image comprising an abstraction of the style image within confines of the silhouette image, the combination loss comprising:
- a content loss between the content feature layer and corresponding one of the reference feature layers;
- and
- a style loss between the pattern feature layers and a corresponding set of the reference feature layers;
- and
- utilizing the output image in a graphic design product.
6 more elements
A system comprising:
- an apparatus comprising a processor and a memory, the processor operable via instructions to:
- process a silhouette image through a convolutional neural network to produce a content feature layer and store the content feature layer in a memory;
- receive a user selection between two different style options, a first option utilizing all sublayers of lower layers of the convolutional neural network for pattern feature layers and performing more than five iterations, the second option utilizing only one sublayer of each lower layer of the convolutional neural network for the pattern feature layers and performing five or fewer iterations;
- process a style image through the convolutional neural network to produce the pattern feature layers according to the user selection and store the pattern feature layer in the memory;
- process a reference image through the convolutional neural network to determine reference feature layers;
- and
- for each of a plurality of iterations, minimize a combination loss of the convolutional neural network to obtain an output image comprising an abstraction of the style image within confines of the silhouette image, the combination loss comprising:
- a content loss between the content feature layer and corresponding one of the reference feature layers;
- and a style loss between the pattern feature layers and a corresponding set of the reference feature layers.
6 more elements
A method comprising:
- inputting a silhouette image and a style image to a convolutional neural network to respectively produce a content feature layer and pattern feature layers;
- receiving a user selection between two different style options, a first option utilizing all sublayers of lower layers of the convolutional neural network for pattern feature layers and performing more than five iterations, the second option utilizing only one sublayer of each lower layer of the convolutional neural network for the pattern feature layers and performing five or fewer iterations, the pattern feature layers being produced based on the user selection;
- combining the content feature layers and the pattern feature layers from the convolutional neural network to obtain an output image, the combination being based on minimizing a content loss between the content feature layer and a corresponding reference feature layer and minimizing a style loss between the pattern feature layers and a corresponding set of other reference feature layers, the output image including an abstraction of the style image within confines of the silhouette image;
- and
- utilizing the output image in a graphic design product.
2 more elements
Dependencies are inferred from the claim text.