# US 8,280,187 B1 · Seam carving and expansion of images with color frequency priority

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,280,187 B1
- Document ID: US8280187B1
- Status: Granted
- Kind: B1
- Application number: 12/184,071
- Filed: 2008-07-31
- Granted: 2012-10-02
- Earliest claimed: 2008-07-31
- Assignees: Adobe Systems Incorporated
- Inventors: Shmuel Avidan; Jen-Chan Chien; Chintan Intwala
- Web page: https://patentgrep.com/patent/US8280187B1

## CPC

- G06T 3/04: Context-preserving transformations, e.g. by using an importance map (panospheric to cylindrical image transformations G06T3/12)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method for expansion and reduction of images in an image editing application uses the frequency of pixels values (e.g., absolute color and/or intensity values) of an input image to determine respective energy values for each pixel of the image. The energy values may vary inversely with the frequency of the pixel values. The energy values may be used to determine one or more low-cost seams of the image to be removed for an image reduction operation or replicated for an image expansion operation, where the cost of each seam is dependent on the energy values of the pixels of the seam. Determining the frequency of pixel values of the image may involve determining the number of pixels having an absolute pixel value in each of a plurality of pixel value ranges, and the absolute pixel values included in each range may be configurable by the user.

## Claims

### Claim 1

A system, comprising:
one or more processors; and
a memory coupled to the one or more processors and storing program instructions executable by the one or more processors to implement:
accessing data representing an image;
determining a frequency of one or more pixel values of the image;
determining a respective energy value for each pixel of the image dependent, at least in part, on said determining a frequency of the one or more pixel values, wherein the energy values indicate the importance of each pixel with respect to resizing operations;
identifying a seam of the image having a lowest cost, wherein the cost of the seam is dependent on the respective energy values of pixels of the seam; and
performing a resizing operation on the image along the identified seam to produce a resized version of the image.

### Claim 2 (depends on 1)

The system of claim 1, wherein the program instructions are further executable to implement repeating said identifying and said performing for a next lowest cost seam until the resized version of the image meets a specified target size or a specified target aspect ratio.

### Claim 3 (depends on 1)

The system of claim 1,
wherein the resizing operation comprises a reduction operation; and
wherein said performing comprises removing the pixels of the seam.

### Claim 4 (depends on 1)

The system of claim 1,
wherein the resizing operation comprises an expansion operation; and
wherein said performing comprises replicating the pixels of the seam and inserting the replicated pixels into the image.

### Claim 5 (depends on 1)

The system of claim 1, wherein the cost of the seam is dependent on one or more of: a sum of the respective energy values of all pixels of the seam, an average of the respective energy values of all pixels of the seam, a weighted average of the respective energy values of all pixels of the seam, or a maximum value of the respective energy values of all pixels of the seam.

### Claim 6 (depends on 1)

The system of claim 1, wherein said identifying a seam of the image having a lowest cost comprises identifying a shortest path across the image.

### Claim 7 (depends on 1)

The system of claim 1, wherein said identifying a seam of the image having a lowest cost comprises applying a dynamic programming technique.

### Claim 8 (depends on 1)

The system of claim 1, wherein said determining a frequency of one or more pixel values of the image comprises determining a number of pixels having an absolute pixel value in each of a plurality of pixel value ranges.

### Claim 9 (depends on 8)

The system of claim 8, wherein the program instructions are further executable to implement configuring the absolute pixel values that are to be included in each of the plurality of pixel value ranges.

### Claim 10 (depends on 9)

The system of claim 9, wherein said configuring is dependent on user input.

### Claim 11 (depends on 1)

The system of claim 1, wherein the respective energy values of each of the pixels of the image vary inversely with the determined frequency of the one or more pixel values.

### Claim 12 (depends on 1)

The system of claim 1, wherein said determining a respective energy value is further dependent on one or more of: a difference between two or more pixel values, a derivative of two or more pixel values, an average of two or more pixel values, a maximum of two or more pixel values, or a minimum of two or more pixel values.

### Claim 13 (depends on 1)

The system of claim 1, wherein each of the pixel values represents one or more of: an absolute color value or an absolute intensity value.

### Claim 14 (depends on 1)

The system of claim 1, wherein the one or more processors comprise at least one of: a general-purpose central processing unit (CPU), or a graphics processing unit (GPU).

### Claim 15

A non-transitory computer-readable storage medium, storing program instructions computer-executable to implement:
accessing data representing an image;
determining a frequency of one or more pixel values of the image;
determining a respective energy value for each pixel of the image dependent, at least in part, on said determining a frequency of the one or more pixel values, wherein the energy values indicate the importance of each pixel with respect to resizing operations;
identifying a seam of the image having a lowest cost, wherein the cost of the seam is dependent on the respective energy values of pixels of the seam; and
performing a resizing operation on the image along the identified seam to produce a resized version of the image.

### Claim 16 (depends on 15)

The storage medium of claim 15, wherein the program instructions are further executable to implement repeating said identifying and said performing for a next lowest cost seam until the resized version of the image meets a specified target size or a specified target aspect ratio.

### Claim 17 (depends on 15)

The storage medium of claim 15,
wherein the resizing operation comprises a reduction operation; and
wherein said performing comprises removing the pixels of the seam.

### Claim 18 (depends on 15)

The storage medium of claim 15,
wherein the resizing operation comprises an expansion operation; and
wherein said performing comprises replicating the pixels of the seam and inserting the replicated pixels into the image.

### Claim 19 (depends on 15)

The storage medium of claim 15, wherein the cost of the seam is dependent on one or more of: a sum of the respective energy values of all pixels of the seam, an average of the respective energy values of all pixels of the seam, a weighted average of the respective energy values of all pixels of the seam, or a maximum value of the respective energy values of all pixels of the seam.

### Claim 20 (depends on 15)

The storage medium of claim 15, wherein said identifying a seam of the image having a lowest cost comprises one or more of: identifying a shortest path across the image or applying a dynamic programming technique.

### Claim 21 (depends on 15)

The storage medium of claim 15, wherein said determining a frequency of one or more pixel values of the image comprises determining a number of pixels having an absolute pixel value in each of a plurality of pixel value ranges.

### Claim 22 (depends on 21)

The storage medium of claim 21, wherein the program instructions are further executable to implement configuring the absolute pixel values that are to be included in each of the plurality of pixel value ranges.

### Claim 23 (depends on 15)

The storage medium of claim 15, wherein the respective energy values of each of the pixels of the image vary inversely with the determined frequency of the one or more pixel values.

### Claim 24 (depends on 15)

The storage medium of claim 15, wherein said determining a respective energy value is further dependent on one or more of: a difference between two or more pixel values, a derivative of two or more pixel values, an average of two or more pixel values, a maximum of two or more pixel values, or a minimum of two or more pixel values.

### Claim 25 (depends on 15)

The storage medium of claim 15, wherein each of the pixel values represents one or more of: an absolute color value or an absolute intensity value.

### Claim 26

A computer-implemented method, comprising:
accessing data representing an image;
determining a frequency of one or more pixel values of the image;
determining a respective energy value for each pixel of the image dependent, at least in part, on said determining a frequency of the one or more pixel values, wherein the energy values indicate the importance of each pixel with respect to resizing operations;
identifying a seam of the image having a lowest cost, wherein the cost of the seam is dependent on the respective energy values of pixels of the seam; and
performing a resizing operation on the image along the identified seam to produce a resized version of the image.

### Claim 27 (depends on 26)

The method of claim 26, further comprising repeating said identifying and said performing for a next lowest cost seam until the resized version of the image meets a specified target size or a specified target aspect ratio.

### Claim 28 (depends on 26)

The method of claim 26,
wherein the resizing operation comprises a reduction operation; and
wherein said performing comprises removing the pixels of the seam.

### Claim 29 (depends on 26)

The method of claim 26,
wherein the resizing operation comprises an expansion operation; and
wherein said performing comprises replicating the pixels of the seam and inserting the replicated pixels into the image.

### Claim 30 (depends on 26)

The method of claim 26, wherein the cost of the seam is dependent on one or more of: a sum of the respective energy values of all pixels of the seam, an average of the respective energy values of all pixels of the seam, a weighted average of the respective energy values of all pixels of the seam, or a maximum value of the respective energy values of all pixels of the seam.

### Claim 31 (depends on 26)

The method of claim 26, wherein said identifying a seam of the image having a lowest cost comprises one or more of: identifying a shortest path across the image or applying a dynamic programming technique.

### Claim 32 (depends on 26)

The method of claim 26, wherein said determining a frequency of one or more pixel values of the image comprises determining a number of pixels having an absolute pixel value in each of a plurality of pixel value ranges.

### Claim 33 (depends on 32)

The method of claim 32, further comprising determining the absolute pixel values to be included in each of the plurality of pixel value ranges dependent on user input.

### Claim 34 (depends on 26)

The method of claim 26, wherein the respective energy values of each of the pixels of the image vary inversely with the determined frequency of the one or more pixel values.

### Claim 35 (depends on 26)

The method of claim 26, wherein said determining a respective energy value is further dependent on one or more of: a difference between two or more pixel values, a derivative of two or more pixel values, an average of two or more pixel values, a maximum of two or more pixel values, or a minimum of two or more pixel values.

### Claim 36 (depends on 26)

The method of claim 26, wherein each of the pixel values represents one or more of: an absolute color value or an absolute intensity value.
