Texture generation using multimodal embeddings

A system comprising:

• at least one processor;
• and
• at least one memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
• storing, in a multimodal memory, interaction data representing use of one or more interaction functions including data in different modalities;
• detecting an object depicted in an image captured by an interaction client;
• generating, by a machine learning model, a prompt based on the object depicted in the image and the interaction data in the multimodal memory;
• generating an artificial texture based on the prompt;
• modifying a texture of the object depicted in the image using the artificial texture that has been generated based on the prompt;
• accessing a fashion item template based on the prompt;
• providing the fashion item template to the machine learning model to populate the fashion item template with the artificial texture;
• and
• overlaying the fashion item template populated with the artificial texture on the object depicted in the image.

A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

• storing, in a multimodal memory, interaction data representing use of one or more interaction functions including data in different modalities;
• detecting an object depicted in an image captured by an interaction client;
• generating, by a machine learning model, a prompt based on the object depicted in the image and the interaction data in the multimodal memory;
• generating an artificial texture based on the prompt;
• modifying a texture of the object depicted in the image using the artificial texture that has been generated based on the prompt;
• accessing a fashion item template based on the prompt;
• providing the fashion item template to the machine learning model to populate the fashion item template with the artificial texture;
• and
• overlaying the fashion item template populated with the artificial texture on the object depicted in the image.