# US 2013/0254688 A1 · Content Aware Image Editing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0254688 A1
- Document ID: US20130254688A1
- Status: Published application
- Kind: A1
- Application number: 13/740,814
- Filed: 2013-01-14
- Published: 2013-09-26
- Earliest claimed: 2012-03-20
- Assignees: Adobe Systems Incorporated
- Applicants: Adobe Systems Incorporated
- Inventors: Elya Shechtman; Dan Goldman
- Web page: https://patentgrep.com/patent/US20130254688A1

## CPC

- G06T 11/60: Creating or editing images; Combining images with text
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0486: Drag-and-drop
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 7/90: Determination of colour characteristics
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10024: Color image
- G06T 2207/20104: Interactive definition of region of interest [ROI]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An image is displayed using a computer system. The image includes contents that have a visible feature therein at a first location. A first input is received that includes a user movement of at least the visible feature from the first location. During the user movement, the first location is synthesized with content from where the visible feature is currently located. A second input is received that specifies an end of the user movement at a second location. A source area in the image is identified. The method further includes identifying additional contents within the source area. The additional contents are identified using a patch-based optimization algorithm on the image. The method further includes updating the image to have the additional contents at least in the first location.

## Claims

### Claim 1

A computer-implemented method for modifying an image, the method comprising:
displaying an image using a computer system, the image comprising contents that have a visible feature therein at a first location;
receiving a first input comprising a user movement of at least the visible feature from the first location;
synthesizing, during the user movement, the first location with content from where the visible feature is currently located;
receiving a second input that specifies an end of the user movement at a second location;
identifying a source area in the image that includes at least the second location and an area around the first location;
identifying additional contents for at least the first location within the source area, the additional contents identified using a patch-based optimization algorithm on the image; and
updating the image to have the additional contents at least in the first location.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first location, after the first input is received, comprises a hole in the image and wherein the updated image is generated in a hole-synthesizing process.

### Claim 3 (depends on 1)

The method of claim 1, wherein updating the image comprises deleting the visible feature from the first location.

### Claim 4 (depends on 1)

The method of claim 1, wherein the image is updated to have at least part of the additional contents in the area around the first location.

### Claim 5 (depends on 1)

The method of claim 1, further comprising generating a source mask for the image having a first color for the source area and a second color for the rest of the source mask.

### Claim 6 (depends on 1)

The method of claim 1, further comprising identifying a synthesis area in the image that includes the first location.

### Claim 7 (depends on 6)

The method of claim 6, further comprising generating a synthesis mask for the image having a first color for the synthesis area and a second color for the rest of the synthesis mask.

### Claim 8

A computer-implemented method for modifying an image, the method comprising:
displaying an image using a computer system, the image comprising contents that have a visible feature therein at a first location;
receiving a first input comprising a user movement of at least the visible feature from the first location;
synthesizing, during the user movement, the first location with content from where the visible feature is currently located;
receiving a second input that specifies an end of the user movement at a second location;
identifying a source area in the image that includes at least the first location, the second location, an area around the first location, and an area around the second location;
identifying a synthesis area that includes the second location;
identifying additional contents for the synthesis area within the source area, the additional contents identified using a patch-based optimization algorithm on the image; and
updating the image to have the additional contents in the synthesis area.

### Claim 9 (depends on 8)

The method of claim 8, wherein the updated image is generated in an extend process in which the visible feature is extended from a first appearance to a second appearance.

### Claim 10 (depends on 8)

The method of claim 8, further comprising updating the image to have the visible feature in the second location.

### Claim 11 (depends on 9)

The method of claim 9, wherein the first location and the second location do not overlap.

### Claim 12 (depends on 8)

The method of claim 8, wherein the synthesis area comprises the second location with an interior portion removed therefrom.

### Claim 13

A computer-implemented method for modifying an image, the method comprising:
displaying an image using a computer system, the image comprising contents that have a visible feature therein at a first location;
receiving a first input comprising a user movement of at least the visible feature from the first location;
synthesizing, during the user movement, the first location with content from where the visible feature is currently located;
receiving a second input that specifies an end of the user movement at a second location;
identifying a source area in the image that includes at least the first location, the second location, an area around the first location, and an area around the second location;
identifying a synthesis area that includes at least a portion of the first location and at least a portion of the second location;
identifying additional contents for at the synthesis area within the source area, the additional contents identified using a patch-based optimization algorithm on the image; and
updating the image to have the additional contents at least in the synthesis area.

### Claim 14 (depends on 13)

The method of claim 13, wherein the updated image is generated in an extend process in which the visible feature is extended from a first appearance to a second appearance.

### Claim 15 (depends on 14)

The method of claim 14, wherein the first location and the second location overlap.

### Claim 16 (depends on 13)

The method of claim 13, wherein the updated image is generated in a move process in which the visible feature is moved from the first location to the second location.

### Claim 17 (depends on 16)

The method of claim 16, wherein the source area does not include the synthesis area.

### Claim 18

A computer-readable medium have stored therein instructions that, when executed, perform a method for modifying an image comprising:
displaying an image using a computer system, the image comprising contents that have a visible feature therein at a first location;
receiving a first input comprising a user movement of at least the visible feature from the first location;
synthesizing, during the user movement, the first location with content from where the visible feature is currently located;
receiving a second input that specifies an end of the user movement at a second location;
identifying a source area in the image that includes at least the second location and an area around the first location;
identifying additional contents for at least the first location within the source area, the additional contents identified using a patch-based optimization algorithm on the image; and
updating the image to have the additional contents at least in the first location.

### Claim 19 (depends on 18)

The computer-readable medium of claim 18, the method further comprising identifying a synthesis area in the image that includes the first location.

### Claim 20

A system comprising:
one or more processors configured to:
provide, to an output device, an image comprising contents that have a visible feature therein at a first location;
receive, from an input device, a first input comprising a user movement of at least the visible feature from the first location;
synthesize, during the user movement, the first location with content from where the visible feature is currently located;
receive a second user input that specifies an end of the user movement at a second location;
identify a source area in the image that includes at least the second location and an area around the first location;
identify additional contents for at least the first location within the source area, the additional contents identified using a patch-based optimization algorithm on the image; and
provide, to the output device, an update to the image having the additional contents at least in the first location.
