# US 11,809,688 B1 · Interactive prompting system for multimodal personalized content generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,809,688 B1 - Document ID: US11809688B1 - Status: Granted - Kind: B1 - Application number: 18/203,524 - Filed: 2023-05-30 - Granted: 2023-11-07 - Earliest claimed: 2023-02-08 - Assignees: Typeface Inc. - Applicants: Typeface Inc. - Inventors: Abhay Parasnis; Anish Pimpley; Kang Chen; Hari Srinivasan; Jonathan Moreira; Vishal Sood; Yue Ning - Web page: https://patentgrep.com/patent/US11809688B1 ## CPC - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/04812: Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 40/103: Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191) - G06F 40/186: Templates - G06F 40/216: using statistical methods - G06F 40/30: Semantic analysis - G06F 40/44: Statistical methods, e.g. probability models - G06F 40/56: Natural language generation - G06N 3/0475: Generative networks - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/77: Retouching; Inpainting; Scratch removal - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 11/00: Two-dimensional [2D] image generation - G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/20084: Artificial neural networks [ANN] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods, systems, and computer programs are presented for providing a prompt tool with interactive entry. One method includes operations for providing a multimodal prompt tool for entering textual description of an item to be generated, and detecting an input that is one of a special character entered in the textual description or a mouse action requesting assistance. Furthermore, a menu is presented with options for the item to be generated, and a list of products, previously added to a data store, is obtained. Further, the method includes providing the list of products for selection; in response to a selection of a product from the list of products, including text associated with the selected product in the textual description; and detecting submittal of the textual description. Further, the textual description is entered as input to a generative artificial intelligence (GAI) tool, and causing presentation of items generated by the GAI tool. ## Claims ### Claim 1 A computer-implemented method comprising: providing, in a user interface (UI), a prompt tool for entering textual description of an item to be generated, wherein the prompt tool is interactive and configured to analyze the textual description to determine a type of item to be created, the item being one of an output text or an output image based on the textual description; detecting, by the prompt tool, a first input while interacting with the prompt tool, the first input being one of a special character entered in the textual description or a mouse action requesting assistance; in response to the first input, presenting a menu in the UI with options for the item to be generated; in response to detecting a selection of asset in the menu, obtaining a list of products previously added to a data store; providing the list of products for selection in the UI; in response to a selection of a product from the list of products, including text associated with the selected product in the textual description; detecting, by the prompt tool, submittal of the textual description; entering the textual description as input to a generative artificial intelligence (GAI) tool; and causing presentation in the UI of one or more items generated by the GAI tool. ### Claim 2 (depends on 1) The method as recited in claim 1, wherein including text associated with the selected product in the textual description comprises: utilizing special formatting for the included text in the textual description that has different formatting from text received from a user in the prompt tool. ### Claim 3 (depends on 1) The method as recited in claim 1, wherein the prompt tool is a guided experience following a rule-based flow to limit options presented based on input received. ### Claim 4 (depends on 1) The method as recited in claim 1, further comprising: detecting that the selected asset is associated with a custom model for the GAI tool that has been trained with data associated with the selected asset; and utilizing the custom model as the GAI tool for generating the one or more items. ### Claim 5 (depends on 1) The method as recited in claim 1, wherein the prompt tool provides an option to derive multimodal content from other pieces of content generated with the prompt tool. ### Claim 6 (depends on 1) The method as recited in claim 1, wherein the menu includes options comprising at least one of content type, asset, overlay type, camera scene, camera lens, camera film type, camera type, lighting style, color theme, image style, or photography style. ### Claim 7 (depends on 6) The method as recited in claim 6, wherein options for content type comprise at least one of photograph, drawing, sketch, or 3D render. ### Claim 8 (depends on 6) The method as recited in claim 6, wherein options for image style comprise at least one realistic, oil painting, pencil art, concept drawing, surrealism, matte, elegant, illustration, digital painting, graffiti, chalk, watercolor, beautiful, most beautiful image ever seen, trending, or epic composition. ### Claim 9 (depends on 6) The method as recited in claim 6, wherein options for subject comprise at least one of product, person, animal, or object. ### Claim 10 (depends on 6) The method as recited in claim 6, wherein options for lighting comprise at least one of accent lighting, ambient lighting, backlight, blacklight, blinding light, candlelight, concert lighting, crepuscular rays, direct sunlight, dusk, Edison bulb, electric arc, fire, fluorescent, glowing, glowing radioactively, glow-stick, lava glow, moonlight, natural lighting, neon lamp, nightclub lighting, nuclear waste glow, quantum dot display, spotlight, strobe, sunlight, ultraviolet, dramatic lighting, dark lighting, soft lighting, or gloomy. ### Claim 11 (depends on 6) The method as recited in claim 6, wherein options for context comprise at least one of indoor, outdoor, at night, sunset, in a park, or bridge in background. ### Claim 12 (depends on 6) The method as recited in claim 6, wherein options for lens comprise at least one of ultra wide-angle, wide angle, aerial view, massive scale, street level view, landscape, panoramic, bokeh, fisheye, dutch angle, low angle, extreme long-shot, long shot, close-up, extreme close-up, highly detailed, depth of field, 4k, 8k, ultra realistic, studio quality, or octane render. ### Claim 13 (depends on 6) The method as recited in claim 6, wherein options for details comprise at least one of highly detailed, grainy, realistic, unreal engine, octane render, bokeh, Vray, Houdini render, quixel megascans, depth of field, Arnold render, 8k, raytracing, CGI, lumen reflections, cgsociety, ultra realistic, volumetric fog, overglaze, analog photo, polaroid, 100 mm, film photography, DSLR, cinema4d, or studio quality. ### Claim 14 A system comprising: a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising: providing, in a user interface (UI), a prompt tool for entering textual description of an item to be generated, wherein the prompt tool is interactive and configured to analyze the textual description to determine a type of item to be created, the item being one of an output text or an output image based on the textual description; detecting, by the prompt tool, a first input while interacting with the prompt tool, the first input being one of a special character entered in the textual description or a mouse action requesting assistance; in response to the first input, presenting a menu in the UI with options for the item to be generated; in response to detecting a selection of asset in the menu, obtaining a list of products previously added to a data store; providing the list of products for selection in the UI; in response to a selection of a product from the list of products, including text associated with the selected product in the textual description; detecting, by the prompt tool, submittal of the textual description; entering the textual description as input to a generative artificial intelligence (GAI) tool; and causing presentation in the UI of one or more items generated by the GAI tool. ### Claim 15 (depends on 14) The system as recited in claim 14, wherein including text associated with the selected product in the textual description comprises: utilizing special formatting for the included text in the textual description that has different formatting from text received from a user in the prompt tool. ### Claim 16 (depends on 14) The system as recited in claim 14, wherein the prompt tool is a guided experience following a rule-based flow to limit options presented based on input received. ### Claim 17 (depends on 14) The system as recited in claim 14, wherein the instructions further cause the one or more computer processors to perform operations comprising: detecting that the selected asset is associated with a custom model for the GAI tool that has been trained with data associated with the selected asset; and utilizing the custom model as the GAI tool for generating the one or more items. ### Claim 18 (depends on 14) The system as recited in claim 14, wherein the prompt tool provides an option to derive multimodal content from other pieces of content generated with the prompt tool. ### Claim 19 A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising: providing, in a user interface (UI), a prompt tool for entering textual description of an item to be generated, wherein the prompt tool is interactive and configured to analyze the textual description to determine a type of item to be created, the item being one of an output text or an output image based on the textual description; detecting, by the prompt tool, a first input while interacting with the prompt tool, the first input being one of a special character entered in the textual description or a mouse action requesting assistance; in response to the first input, presenting a menu in the UI with options for the item to be generated; in response to detecting a selection of asset in the menu, obtaining a list of products previously added to a data store; providing the list of products for selection in the UI; in response to a selection of a product from the list of products, including text associated with the selected product in the textual description; detecting, by the prompt tool, submittal of the textual description; entering the textual description as input to a generative artificial intelligence (GAI) tool; and causing presentation in the UI of one or more items generated by the GAI tool. ### Claim 20 (depends on 19) The non-transitory machine-readable storage medium as recited in claim 19, wherein including text associated with the selected product in the textual description comprises: utilizing special formatting for the included text in the textual description that has different formatting from text received from a user in the prompt tool.