# US 2026/0227974 A1 · Editing User Interfaces using Sketches > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2026/0227974 A1 - Document ID: US20260227974A1 - Status: Published application - Kind: A1 - Application number: 19/634,378 - Filed: 2026-03-31 - Published: 2026-08-06 - Earliest claimed: 2022-09-10 - Assignees: Tweak Commerce Inc. - Applicants: Tweak Commerce Inc. - Inventors: Nikolas Louis Ciminelli; Tom Zeev Jacob Palny; Ron ZASS - Web page: https://patentgrep.com/patent/US20260227974A1 ## CPC - G06T 11/60: Creating or editing images; Combining images with text - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/04847: Interaction techniques to control parameter settings, e.g. interaction with sliders or dials - G06F 3/0487: using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser - G06F 8/36: Software reuse - G06F 8/38: for implementing user interfaces - G06F 8/71: Version control (security arrangements therefor G06F21/57); Configuration management - G06F 9/451: Execution arrangements for user interfaces - G06F 40/253: Grammatical analysis; Style critique - G06F 40/30: Semantic analysis - G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) - G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation - G06V 10/945: {User interactive design; Environments; Toolboxes} - G06V 30/19173: {Classification techniques} - G06V 30/422: Technical drawings; Geographical maps - H04L 67/306: User profiles - H04W 4/02: Services making use of location information - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0475: Generative networks - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06N 20/20: Ensemble learning - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems, methods and non-transitory computer readable media for editing user interfaces from free text and/or sketches are provided. Digital data associated with a design of a user interface may be received. A presentation of the user interface based on the design may be caused. A selection of a portion of the user interface may be received. A textual input in a natural language and/or a sketch may be received. The textual input and/or the sketch may be analyzed to determine at least one change to a portion of the design of the user interface corresponding to the selected portion of the user interface. The determined at least one change may be implemented to generate a modified design of the user interface. A second presentation of the user interface based on the modified design may be caused. ## Claims ### Claim 1 A non-transitory computer readable medium storing computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for editing user interfaces using sketches, the operations comprising: receiving digital data associated with a design of a user interface, the user interface includes a plurality of elements; using the digital data to cause a first presentation of the user interface to an individual, the first presentation is based on the design; receiving from the individual a selection of a portion of the user interface, the selected portion of the user interface includes at least two but not all elements of the plurality of elements; receiving from the individual a sketch, the sketch is associated with a desire of the individual to modify a portion of the design of the user interface corresponding to the selected portion of the user interface; analyzing the sketch to determine at least one change to the portion of the design of the user interface, wherein the at least one change comprises changing a level of details associated with at least part of the user interface; implementing the determined at least one change to generate a modified design of the user interface; and causing a second presentation of the user interface to the individual, the second presentation is based on the modified design. ### Claim 2 (canceled) ### Claim 3 (canceled) ### Claim 4 (depends on 1) The non-transitory computer readable medium of claim 1, wherein operations further comprise: selecting a new location for a particular element of the elements included in the selected portion of the user interface, wherein the new location is a relative spatial location of the particular element with respect to a second element included in the user interface; and including a change of a location of the particular element to the new location in the at least one change. ### Claim 5 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: selecting a new timing for a particular animation associated with a particular element of the elements included in the selected portion of the user interface, wherein the new timing defines when the particular animation occurs relative to a user action; and including a change to a timing of the particular animation based on the new timing in the at least one change. ### Claim 6 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: selecting a new color scheme for a particular element of the elements included in the selected portion of the user interface; and including a change of a color scheme of the particular element to the new color scheme in the at least one change. ### Claim 7 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: selecting a new layout for the elements included in the selected portion of the user interface, wherein the selected new layout is at least one of a symmetrical structure or a linear structure; and including in the at least one change a change of a layout of the elements included in the selected portion of the user interface to the new layout. ### Claim 8 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the elements included in the selected portion of the user interface includes at least a first element and a second element, and the operations further comprise: determining a change to a spatial relation between the first element and the second element, wherein the change to a spatial relation includes moving only one of the first element or the second element to a target location; and including the determined change to the spatial relation between the first element and the second element in the at least one change. ### Claim 9 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: after the second presentation, receiving from the individual a second sketch, the second sketch is associated with a second desire of the individual to modify the portion of the design of the user interface corresponding to the selected portion of the user interface; analyzing the second sketch to determine a second at least one change to the portion of the design of the user interface; and implementing the determined second at least one change to generate a second modified design of the user interface. ### Claim 10 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: after the second presentation, receiving from the individual an input in a natural language, the input includes an interjection; based on the interjection, determining a degree of success in the determination of the at least one change to the portion of the design of the user interface; and based on the determined degree of success, generating a second modified design of the user interface. ### Claim 11 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: receiving from the individual an input in a natural language; and analyzing the input in the natural language to select the portion of the user interface. ### Claim 12 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the determination of the at least one change to the portion of the design of the user interface is further based on a characteristic of a target audience, wherein the characteristic comprises a behavior pattern associated with the target audience. ### Claim 13 (depends on 12) The non-transitory computer readable medium of claim 12, wherein the operations further comprise analyzing historic activities of individuals associated with the target audience to determine the characteristic of the target audience. ### Claim 14 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the determination of the at least one change to the portion of the design of the user interface is further based on a particular design style, and wherein the operations further comprise analyzing at least one other user interface to determine the particular design style. ### Claim 15 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise using a machine learning model to analyze the sketch to determine the at least one change to the portion of the design of the user interface. ### Claim 16 (canceled) ### Claim 17 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: receiving from the individual an input in a natural language, the input in the natural language is associated with the desire of the individual to modify the portion of the design of the user interface corresponding to the selected portion of the user interface; and analyzing the input in the natural language and the sketch to determine the at least one change to the portion of the design of the user interface. ### Claim 18 (depends on 17) The non-transitory computer readable medium of claim 17, wherein the sketch includes a plurality of pixel values, and the operations further comprise: calculating a convolution of at least part of the pixel values of the sketch to obtain a result value; identifying a first mathematical object in a mathematical space, the first mathematical object corresponds to a word of the input in the natural language; calculating a function of the result value and the first mathematical object to obtain a second mathematical object in the mathematical space; and basing the determination of the at least one change to the portion of the design of the user interface on the second mathematical object. ### Claim 19 A system for editing user interfaces using sketches, the system comprising at least one processing unit configured to perform operations, the operations comprise: receiving digital data associated with a design of a user interface, the user interface includes a plurality of elements; using the digital data to cause a first presentation of the user interface to an individual, the first presentation is based on the design; receiving from the individual a selection of a portion of the user interface, the selected portion of the user interface includes at least two but not all elements of the plurality of elements; receiving from the individual sketch, the sketch is associated with a desire of the individual to modify a portion of the design of the user interface corresponding to the selected portion of the user interface; analyzing the sketch to determine at least one change to the portion of the design of the user interface, wherein the at least one change comprises changing a level of details associated with at least part of the user interface; implementing the determined at least one change to generate a modified design of the user interface; and causing a second presentation of the user interface to the individual, the second presentation is based on the modified design. ### Claim 20 A method for editing user interfaces using sketches, the method comprising: receiving digital data associated with a design of a user interface, the user interface includes a plurality of elements; using the digital data to cause a first presentation of the user interface to an individual, the first presentation is based on the design; receiving from the individual a selection of a portion of the user interface, the selected portion of the user interface includes at least two but not all elements of the plurality of elements; receiving from the individual a sketch, the sketch is associated with a desire of the individual to modify a portion of the design of the user interface corresponding to the selected portion of the user interface; analyzing the sketch to determine at least one change to the portion of the design of the user interface, wherein the at least one change comprises changing a level of details associated with at least part of the user interface; implementing the determined at least one change to generate a modified design of the user interface; and causing a second presentation of the user interface to the individual, the second presentation is based on the modified design. ### Claim 21 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: after the second presentation, receiving from the individual an input in a natural language, the input in the natural language is associated with a second desire of the individual to modify the portion of the design of the user interface corresponding to the selected portion of the user interface; analyzing the second input in the natural language to determine a second at least one change to the portion of the design of the user interface; and implementing the determined second at least one change to generate a second modified design of the user interface. ### Claim 22 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: selecting a new size for a particular element of the elements included in the selected portion of the user interface; and including a change of a size of the particular element to the new size in the at least one change. ### Claim 23 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the operations further comprise: selecting a particular element of the elements included in the selected portion of the user interface; and based on the selection of the particular element, including a change to the particular element in the at least one change.