Style matching using layer-wise masks

A method comprising:

• encoding a content prompt and a style prompt to obtain a content embedding and a style embedding, respectively;
• identifying a first content mask, a second content mask, a first style mask, and a second style mask, wherein the first content mask and the first style mask correspond to a first layer of an image generation model and the second content mask and the second style mask correspond to a second layer of the image generation model;
• applying the first content mask and the second content mask to the content embedding and applying the first style mask and the second style mask to the style embedding to obtain a first weighted content embedding, a second weighted content embedding, a first weighted style embedding and a second weighted style embedding;
• and
• generating, using the image generation model, a synthetic image based on the first of weighted content embedding, the second weighted content embedding, the first weighted style embedding and the second weighted style embedding, wherein the first layer of the image generation model takes the first weighted content embedding and the first weighted style embedding as input and the second layer of the image generation model takes the second weighted content embedding and the second weighted style embedding as input.

An apparatus comprising:

• at least one processor;
• at least one memory storing instructions executable by the at least one processor;
• and
• the apparatus further comprising an image generation model comprising parameters stored in the at least one memory and configured to generate a synthetic image based on a weighted content embedding and a weighted style embedding, wherein the weighted content embedding is obtained by applying a content mask to a content embedding, and wherein the weighted style embedding is obtained by applying a style mask to a style embedding.