# US 12,045,735 B1 · Interactive template for multimodal content generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,045,735 B1 - Document ID: US12045735B1 - Status: Granted - Kind: B1 - Application number: 18/203,537 - Filed: 2023-05-30 - Published: 2024-08-08 - Granted: 2024-07-23 - Earliest claimed: 2023-02-08 - Assignees: Typeface Inc. - Applicants: Typeface Inc. - Inventors: Abhay Parasnis; Kang Chen; Hari Srinivasan; Jonathan Moreira; Vishal Sood; Yue Ning - Web page: https://patentgrep.com/patent/US12045735B1 ## CPC - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 40/103: Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191) - G06F 40/186: Templates - G06F 40/216: using statistical methods - G06F 40/30: Semantic analysis - G06F 40/44: Statistical methods, e.g. probability models - G06F 40/56: Natural language generation - G06N 3/0475: Generative networks - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/77: Retouching; Inpainting; Scratch removal - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 11/00: Two-dimensional [2D] image generation - G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/20084: Artificial neural networks [ANN] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods, systems, and computer programs are presented for generating multimodal content utilizing multimodal templates. One method includes presenting, in a user interface (UI), a template-selection option with one or more templates. Each template comprises a sequence of operations, where each operation comprises a prompt for creating items using generative artificial intelligence (GAI) tools. Further, each operation in the template is multimodal to be configurable to create text and configurable to create one or more images. The method further includes detecting a selection of a template in the UI. For each operation in the selected template, perform operations comprising: presenting, in the UI, the prompt associated with the operation; in response to receiving an input for the prompt, selecting a GAI tool based on a mode of the operation; providing the input to the selected GAI tool to generate the item; and presenting, in the UI, the generated item. ## Claims ### Claim 1 A computer-implemented method comprising: presenting, in a user interface (UI), a template-selection option with one or more multimodal templates to generate multimodal content that comprises a plurality of items, each multimodal template comprising a sequence of operations, each operation of the multimodal template comprising a prompt for creating an item using generative artificial intelligence (GAI) tools, each prompt comprising descriptive text for one of the GAI tools to create the item associated with the operation, each operation in the one or more multimodal templates being multimodal to be configurable to create text and to be configurable to create one or more images by one of the GAI tools, each GAI tool configured to generate new content based on a textual description in the prompt; detecting a selection of a multimodal template in the UI; and for each operation in the selected multimodal template: presenting, in the UI, the prompt for the GAI tool associated with the operation, the prompt including the descriptive text to create the item associated with the operation; in response to receiving an input for the prompt in the UI, selecting a selected GAI tool from the GAI tools based on a mode of the operation; providing the input to the selected GAI tool to generate the item associated with the operation; and presenting, in the UI, the generated item. ### Claim 2 (depends on 1) The method as recited in claim 1, wherein each multimodal template is configurable to be associated with an asset, wherein providing the input to the selected GAI tool further comprises: obtaining metadata for the asset from an asset catalog; and providing the metadata for the asset to the selected GAI tool. ### Claim 3 (depends on 1) The method as recited in claim 1, further comprising: tracking a plurality of operations that generated the multimodal content; presenting, in the UI, the tracked plurality of operations; and providing, in the UI, an option to generate a generated multimodal template based on the tracked plurality of operations, wherein the generated multimodal template is generated without requiring programming. ### Claim 4 (depends on 1) The method as recited in claim 1, wherein each multimodal template is further configurable to create a video file and configurable to create a sound file. ### Claim 5 (depends on 1) The method as recited in claim 1, wherein the multimodal template is further configurable to include relationships between blocks, wherein a first relationship comprises generating an image based on a generated text, and a second relationship comprises generating text based on an image. ### Claim 6 (depends on 1) The method as recited in claim 1, wherein the operations of the multimodal template include at least one of entering text for a title, selecting a product from a catalog, entering a description of the image, or entering a prompt input for generating text. ### Claim 7 (depends on 1) The method as recited in claim 1, wherein the multimodal template is for generating a post to be posted on Instagram social network, the multimodal template comprising one or more images, a text associated with the one or more images, and one or more hashtags. ### Claim 8 (depends on 1) The method as recited in claim 1, wherein the multimodal template is for generating a post to be posted on Facebook social network, the multimodal template comprising one or more images, a text associated with the one or more images, a headline, and a call-to-action button. ### Claim 9 (depends on 1) The method as recited in claim 1, wherein the multimodal template is a blog-post template for generating a blog post, the blog-post template comprising a title, an introduction text, a transition text, one or more outline headings describing what is covered in the blog post, a summary text, and a call to action. ### Claim 10 (depends on 1) The method as recited in claim 1, wherein the sequence of operations for a blog-post template comprises: generating an outline; generating a title for a blog post; generating an introduction paragraph; generating an image based on a topic of the blog post; generating a transition paragraph; generating an outline for topics to cover in the blog post; expanding the outline topics with one or more paragraphs; generating optional images for each topic; and generating a summary for the blog post. ### Claim 11 (depends on 1) The method as recited in claim 1, wherein the sequence of operations for a sample multimodal template comprises: generating an image; generating a caption for the image; and generating hashtags based on the generated image and the generated caption. ### Claim 12 (depends on 1) The method as recited in claim 1, further comprising: after performing the operations of the selected multimodal template, providing a canvas in the UI in interactive mode to enable refinement of the items generated using the multimodal template, the canvas presenting the plurality of items associated with the multimodal template. ### Claim 13 A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising: presenting, in a user interface (UI), a template-selection option with one or more multimodal templates to generate multimodal content that comprises a plurality of items, each multimodal template comprising a sequence of operations, each operation of the multimodal template comprising a prompt for creating an item using generative artificial intelligence (GAI) tools, each prompt comprising descriptive text for one of the GAI tools to create the item associated with the operation, each operation in the one or more multimodal templates being multimodal to be configurable to create text and to be configurable to create one or more images by one of the GAI tools, each GAI tool configured to generate new content based on a textual description in the prompt; detecting a selection of a selected multimodal template in the UI; and for each operation in the selected multimodal template: presenting, in the UI, the prompt for the GAI tool associated with the operation, the prompt including the descriptive text to create the item associated with the operation; in response to receiving an input for the prompt in the UI, selecting a selected GAI tool from the GAI tools based on a mode of the operation; providing the input to the selected GAI tool to generate the item associated with the operation; and presenting, in the UI, the generated item. ### Claim 14 (depends on 13) The system as recited in claim 13, wherein each multimodal template is configurable to be associated with an asset, wherein providing the input to the selected GAI tool further comprises: obtaining metadata for the asset from an asset catalog; and providing the metadata for the asset to the selected GAI tool. ### Claim 15 (depends on 13) The system as recited in claim 13, wherein the instructions further cause the one or more computer processors to perform operations comprising: tracking a plurality of operations that generated the multimodal content; presenting, in the UI, the tracked plurality of operations; and providing, in the UI, an option to generate a generated multimodal template based on the tracked plurality of operations, wherein the generated multimodal template is generated without requiring programming. ### Claim 16 (depends on 13) The system as recited in claim 13, wherein each multimodal template is further configurable to create a video file and configurable to create a sound file. ### Claim 17 (depends on 13) The system as recited in claim 13, wherein the multimodal template is further configurable to include relationships between blocks, wherein a first relationship comprises generating an image based on a generated text, and a second relationship comprises generating text based on an image. ### Claim 18 A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: presenting, in a user interface (UI), a template-selection option with one or more multimodal templates to generate multimodal content that comprises a plurality of items, each multimodal template comprising a sequence of operations, each operation of the multimodal template comprising a prompt for creating an item using generative artificial intelligence (GAI) tools, each prompt comprising descriptive text for one of the GAI tools to create the item associated with the operation, each operation in the one or more multimodal templates being multimodal to be configurable to create text and to be configurable to create one or more images by one of the GAI tools, each GAI tool configured to generate new content based on a textual description in the prompt; detecting a selection of a multimodal template in the UI; and for each operation in the selected multimodal template: presenting, in the UI, the prompt for the GAI tool associated with the operation, the prompt including the descriptive text to create the item associated with the operation; in response to receiving an input for the prompt in the UI, selecting a selected GAI tool from the GAI tools based on a mode of the operation; providing the input to the selected GAI tool to generate the item associated with the operation; and presenting, in the UI, the generated item. ### Claim 19 (depends on 18) The non-transitory machine-readable storage medium as recited in claim 18, wherein each multimodal template is configurable to be associated with an asset, wherein providing the input to the selected GAI tool further comprises: obtaining metadata for the asset from an asset catalog; and providing the metadata for the asset to the selected GAI tool. ### Claim 20 (depends on 18) The non-transitory machine-readable storage medium as recited in claim 18, wherein the machine further performs operations comprising: tracking a plurality of operations that generated the multimodal content; presenting, in the UI, the tracked plurality of operations; and providing, in the UI, an option to generate a generated multimodal template based on the tracked plurality of operations, wherein the generated multimodal template is generated without requiring programming.