# US 9,436,382 B2 · Natural language image editing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,382 B2 - Document ID: US9436382B2 - Status: Granted - Kind: B2 - Application number: 13/683,278 - Filed: 2012-11-21 - Published: 2014-03-20 - Granted: 2016-09-06 - Earliest claimed: 2012-09-18 - Assignees: Adobe Systems Incorporated - Applicants: Adobe Systems Incorporated - Inventors: Gregg D. Wilensky; Walter W. Chang; Lubomira A. Dontcheva; Gierad P. Laput; Aseem O. Agarwala - Web page: https://patentgrep.com/patent/US9436382B2 ## CPC - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06F 9/451: Execution arrangements for user interfaces - G06T 11/60: Creating or editing images; Combining images with text - G10L 15/1822: {Parsing for meaning understanding} - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Natural language image editing techniques are described. In one or more implementations, a natural language input is converted from audio data using a speech-to-text engine. A gesture is recognized from one or more touch inputs detected using one or more touch sensors. Performance is then initiated of an operation identified from a combination of the natural language input and the recognized gesture. ## Claims ### Claim 1 A method implemented by at least one computing device, the method comprising: receiving a natural language input converted from audio data using a speech-to-text engine; recognizing a gesture from at least one touch input detected using at least one touch sensor; identifying, from a combination of the natural language input and the recognized gesture, an image editing operation to be performed on image data of an image; and performing the identified image editing operation by editing the image data of the image to alter at least one pixel of the image. ### Claim 2 (depends on 1) A method as described in claim 1, wherein the gesture specifies the image editing operation to be performed and the natural language input specifies a subject of the image editing operation. ### Claim 3 (depends on 1) A method as described in claim 1, wherein the at least one touch sensor is configured to implement touchscreen functionality of a display device that displays the image. ### Claim 4 (depends on 1) A method as described in claim 1, further comprising determining a portion of the image that corresponds to the natural language input and the performance of the image editing operation involves the determined portion. ### Claim 5 (depends on 4) A method as described in claim 4, wherein the natural language input involves a name and the determining is performed to define at least part of a boundary of at least a portion of the image that is associated with the name. ### Claim 6 (depends on 5) A method as described in claim 5, wherein the name is a name of a landmark and the determining further comprises identifying the landmark in the image. ### Claim 7 (depends on 1) A method as described in claim 1, wherein at least one of the natural language input or the gesture specifies a degree to which the image editing operation is to be performed. ### Claim 8 (depends on 1) A method as described in claim 1, wherein the image editing operation involves spatial localization. ### Claim 9 (depends on 1) A method as described in claim 1, wherein the image editing operation involves two or more filter operations. ### Claim 10 A method comprising: obtaining a natural language input by a computing device; recognizing a gesture from at least one touch input detected using touchscreen functionality of a display device of the computing device, the display device displaying an image; identifying, from a combination of the natural language input and the recognized gesture, an image editing operation to be performed by the computing device on image data of the image; and performing the identified image editing operation by editing the image data of the image to alter at least one pixel of the image. ### Claim 11 (depends on 10) A method as described in claim 10, wherein the natural language input is converted to text from audio data using a speech-to-text engine. ### Claim 12 (depends on 10) A method as described in claim 10, wherein the natural language input includes text that is manually entered by a user. ### Claim 13 (depends on 10) A method as described in claim 10, wherein at least one of the gesture or the natural language input identifies the image editing operation and the other of the gesture or the natural language input identifies a subject of the image editing operation. ### Claim 14 (depends on 13) A method as described in claim 13, wherein the subject is a portion of the image to which the image editing operation is performed. ### Claim 15 (depends on 10) A method as described in claim 10, wherein the natural language input involves a name, and the identifying further comprises determining a portion of the image that corresponds to the name. ### Claim 16 A system comprising: a speech-to-text engine configured to convert audio data into a natural language input comprising text; a gesture module configured to recognize a gesture from at least one touch input detected using at least one touch sensor; and a natural language processing module configured to: identify, from a combination of the natural language input and the gesture, an image editing operation to be performed on image data of an image; and initiate performance of the image editing operation by a computing device to edit the image data of the image to alter at least one pixel of the image. ### Claim 17 (depends on 16) A system as described in claim 16, wherein at least one of the natural language input or the gesture specifies a degree to which the image editing operation is to be performed. ### Claim 18 (depends on 16) A system as described in claim 16, wherein the speech-to-text engine is configured to form a communication for communication via a network to another speech-to-text engine responsive to a determination that at least a portion of the audio data to be converted to text does not meet a threshold, the communication including the portion of the audio data. ### Claim 19 (depends on 16) A system as described in claim 16, wherein the gesture identifies a base of the image that is to be subject of further processing by an object identification module to determine a boundary of an object that includes the base. ### Claim 20 (depends on 16) A system as described in claim 16, wherein the natural language input involves a name, the natural language processing module further configured to determine a portion of the image that corresponds to the name.