# US 11,928,319 B1 · Interactive canvas tool for multimodal personalized content generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,928,319 B1 - Document ID: US11928319B1 - Status: Granted - Kind: B1 - Application number: 18/203,530 - Filed: 2023-05-30 - Granted: 2024-03-12 - Earliest claimed: 2023-02-08 - Assignees: Typeface Inc. - Applicants: Typeface Inc. - Inventors: Abhay Parasnis; Kang Chen; Hari Srinivasan; Jonathan Moreira; Vishal Sood - Web page: https://patentgrep.com/patent/US11928319B1 ## CPC - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 40/103: Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191) - G06F 40/186: Templates - G06F 40/216: using statistical methods - G06F 40/30: Semantic analysis - G06F 40/44: Statistical methods, e.g. probability models - G06F 40/56: Natural language generation - G06N 3/0475: Generative networks - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/77: Retouching; Inpainting; Scratch removal - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 11/00: Two-dimensional [2D] image generation - G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/20084: Artificial neural networks [ANN] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods, systems, and computer programs are presented for providing an interactive canvas tool to generate multimodal, personalized content. One method includes providing a user interface (UI) for a canvas tool to generate multimodal content. The canvas tool comprises a prompt panel, a variations panel, and a canvas configured to present items of several types, such as a text type and an image type. The method further includes receiving text input including a textual description with instruction for generating an item, where the canvas tool is configured to generate items from the several types. The method further includes providing the text input to a generative artificial intelligence (GAI) tool, and presenting one or more variations based on an output of the GAI tool. The method further includes detecting a selection of one of the variations in the variations panel, and adding the selected variation to the canvas in the UI. ## Claims ### Claim 1 A computer-implemented method comprising: providing a user interface (UI) by a canvas tool to generate multimodal content, the canvas tool comprising a prompt panel, a variations panel, and a canvas panel the canvas panel configured to present items from a plurality of item types, the plurality of item types comprising a text type and an image type; receiving text input in the prompt panel, the text input including a textual description with instruction for generating one or more variations, the canvas tool being configured to generate the one or more variations from each type of the plurality of item types; providing the text input to a generative artificial intelligence (GAI) tool to generate the one or more variations based on the textual description of the text input, the GAI tool configured to generate new content based on the textual description; presenting, in the variations panel, the one or more variations generated by the GAI tool; detecting a selection of a selected variation from the one or more variations in the variations panel; and adding the selected variation to the canvas panel in the UI. ### Claim 2 (depends on 1) The method as recited in claim 1, further comprising: determining a requested item type, from the plurality of item types, based on the received text input; and selecting, based on the requested item type, the GAI tool, from a plurality of GAI tools, for generating the one or more variations. ### Claim 3 (depends on 1) The method as recited in claim 1, wherein the received text input includes a request to generate text, wherein the one or more variations include text descriptions. ### Claim 4 (depends on 1) The method as recited in claim 1, wherein the received text input includes a request to generate a new image, the received text input including the textual description for the generation of the new image, wherein the one or more variations include images. ### Claim 5 (depends on 4) The method as recited in claim 4, further comprising: before presenting the one or more variations, modifying each output of the GAI tool to embed an image of a custom asset from a user database. ### Claim 6 (depends on 1) The method as recited in claim 1, further comprising: repeating the receiving text input and the generating to create new variations; and adding one of the new variations to the canvas panel. ### Claim 7 (depends on 1) The method as recited in claim 1, further comprising: providing an option in the UI to edit text in the canvas panel. ### Claim 8 (depends on 1) The method as recited in claim 1, further comprising: utilizing a template to generate several variations for the canvas panel, the template comprising a plurality of operations, each operation comprising an option to enter a description in the prompt panel to generate the one or more variations. ### Claim 9 (depends on 1) The method as recited in claim 1, wherein the plurality of item types further comprises a video type and an audio type. ### Claim 10 (depends on 1) The method as recited in claim 1, wherein the canvas panel comprises a text variation and an image variation. ### Claim 11 A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising: providing a user interface (UI) by a canvas tool to generate multimodal content, the canvas tool comprising a prompt panel, a variations panel, and a canvas panel, the canvas panel configured to present items from a plurality of item types, the plurality of item types comprising a text type and an image type; receiving text input in the prompt panel, the text input including a textual description with instruction for generating one or more variations, the canvas tool being configured to generate the one or more variations from each type of the plurality of item types; providing the text input to a generative artificial intelligence (GAI) tool to generate the one or more variations based on the textual description of the text input, the GAI tool configured to generate new content based on the textual description; presenting, in the variations panel, the one or more variations generated by the GAI tool; detecting a selection of a selected variation from the one or more variations in the variations panel; and adding the selected variation to the canvas panel in the UI. ### Claim 12 (depends on 11) The system as recited in claim 11, wherein the instructions further cause the one or more computer processors to perform operations comprising: determining a requested item type, from the plurality of item types, based on the received text input; and selecting, based on the requested item type, the GAT tool, from a plurality of GAI tools, for generating the one or more variations. ### Claim 13 (depends on 11) The system as recited in claim 11, wherein the received text input includes a request to generate text, wherein the one or more variations include text descriptions. ### Claim 14 (depends on 11) The system as recited in claim 11, wherein the received text input includes a request to generate a new image, the received text input including the textual description for the generation of the new image, wherein the one or more variations include images. ### Claim 15 (depends on 11) The system as recited in claim 11, wherein the instructions further cause the one or more computer processors to perform operations comprising: providing an option in the UI to edit text in the canvas panel. ### Claim 16 A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: providing a user interface (UI) by a canvas tool to generate multimodal content, the canvas tool comprising a prompt panel, a variations panel, and a canvas panel the canvas panel configured to present items from a plurality of item types, the plurality of item types comprising a text type and an image type; receiving text input in the prompt panel, the text input including a textual description with instruction for generating one or more variations, the canvas tool being configured to generate the one or more variations from each type of the plurality of item types; providing the text input to a generative artificial intelligence (GAI) tool to generate the one or more variations based on the textual description of the text input, the GAI tool configured to generate new content based on the textual description; presenting, in the variations panel, the one or more variations generated by the GAI tool; detecting a selection of a selected variation from the one or more variations in the variations panel; and adding the selected variation to the canvas panel in the UI. ### Claim 17 (depends on 16) The non-transitory machine-readable storage medium as recited in claim 16, wherein the machine further performs operations comprising: determining a requested item type, from the plurality of item types, based on the received text input, and selecting, based on the requested item type, the GAI tool, from a plurality of GAI tools, for generating the one or more variations. ### Claim 18 (depends on 16) The non-transitory machine-readable storage medium as recited in claim 16, wherein the received text input includes a request to generate text, wherein the one or more variations include text descriptions. ### Claim 19 (depends on 16) The non-transitory machine-readable storage medium as recited in claim 16, wherein the received text input includes a request to generate a new image, the received text input including the textual description for the generation of the new image, wherein the one or more variations include images. ### Claim 20 (depends on 16) The non-transitory machine-readable storage medium as recited in claim 16, wherein the machine further performs operations comprising: providing an option in the UI to edit text in the canvas panel.