# US 9,438,769 B1 · Preserving smooth-boundaried objects of an image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,438,769 B1
- Document ID: US9438769B1
- Status: Granted
- Kind: B1
- Application number: 14/806,923
- Filed: 2015-07-23
- Granted: 2016-09-06
- Earliest claimed: 2015-07-23
- Assignees: Hewlett-Packard Development Company, L.P.
- Applicants: Hewlett-Packard Development Company, L.P.
- Inventors: Jian Fan
- Web page: https://patentgrep.com/patent/US9438769B1

## CPC

- H04N 1/4072: {dependent on the contents of the original}
- H04N 1/40062: {Discrimination between different image types, e.g. two-tone, continuous tone}
- H04N 1/40068: {Modification of image resolution, i.e. determining the values of picture elements at new relative positions (H04N1/3935 takes precedence)}
- H04N 1/4092: {Edge or detail enhancement}
- H04N 1/58: Edge or detail enhancement; Noise or error suppression, e.g. colour misregistration correction (H04N1/62 takes precedence)
- H04N 2201/0081: Image reader (H04N2201/0091 - H04N2201/0094 take precedence)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In one example, a method of automatically enhancing a digital image having a smooth-boundaried foreground object. The method processes the image via gradient-based background segmentation at a first scale to generate a first mask. The first mask has a first accuracy. The first mask is used to generate a first background of the image which includes at least a portion of the smooth-boundaried foreground object. The method further processes the first background via gradient-based background segmentation at a smaller second scale to generate a second mask. The second mask has a greater second accuracy. The second mask is used with the first mask to generate a second background of the image which substantially excludes the smooth-boundaried foreground object. The image can be enhanced using the second background so as to preserve the smooth-boundaried foreground object.

## Claims

### Claim 1

A method of automatically enhancing a digital image having a smooth-boundaried foreground object, comprising:
processing the image via gradient-based background segmentation at a first scale to generate a first mask, having a first accuracy, used to generate a first background of the image which includes at least a portion of the smooth-boundaried foreground object;
processing the first background via gradient-based background segmentation at a smaller second scale to generate a second mask, having a greater second accuracy, used with the first mask to generate a second background of the image which substantially excludes the smooth-boundaried foreground object;
enhancing the image using the second background so as to preserve the smooth-boundaried foreground object; and
outputting the enhanced image.

### Claim 2 (depends on 1)

The method of claim 1, wherein processing the image comprises:
segmenting the image to form the first mask, at the first scale, identifying background and non-background regions of the image with the first accuracy;
forming the first background by inpainting the non-background regions of the image identified by the first mask based on a background region of the image identified by the first mask.

### Claim 3 (depends on 2)

The method of claim 2, wherein segmenting the image comprises:
determining gradient magnitude values of pixels of the image;
determining a first global threshold from the gradient magnitude values;
thresholding the gradient magnitude values using the first global threshold to form thresholded gradient magnitude values;
assigning the pixels to groups based on a watershed transform of the thresholded gradient magnitude values; and
denoting each group as a background region or a non-background region based on at least one characteristic of the corresponding group.

### Claim 4 (depends on 1)

The method of claim 1, wherein processing the first background comprises:
downscaling the first background to the smaller second scale;
segmenting the downscaled first background to form a second mask, at the second scale, identifying background and non-background regions of the downscaled first background;
upscaling the second mask to the first scale;
combining the upscaled second mask with the first mask to form a composite mask which locates the smooth-boundaried foreground object with the greater second accuracy; and
forming the second background by inpainting the non-background regions of the image identified by the composite mask based on a background region of the image.

### Claim 5 (depends on 4)

The method of claim 4, wherein the combining comprises:
defining as a background region of the composite mask the background region of the first mask and the background region of the upscaled second mask; and
defining portions of the composite mask other than the background region as non-background regions of the composite mask.

### Claim 6 (depends on 4)

The method of claim 4, wherein segmenting the downscaled first background comprises:
determining gradient magnitude values of pixels of the downscaled first background;
determining a second global threshold from the gradient magnitude values;
reducing the second global threshold by a predetermined factor;
thresholding the gradient magnitude values using the reduced second global threshold to form thresholded gradient magnitude values;
assigning the pixels to groups based on a watershed transform of the thresholded gradient magnitude values; and
denoting the groups as background regions or non-background regions based on at least one characteristic of the corresponding group.

### Claim 7 (depends on 4)

The method of claim 4, comprising:
upscaling the second background to a scale of an original version of the image;
deriving spatially-variant gains for each pixel of the original version from the upscaled second background; and
applying the spatially-variant gains to each pixel of the original version so as to form an enhanced original version that corrects for color cast and uneven illumination in the original version.

### Claim 8 (depends on 7)

The method of claim 7, comprising:
upscaling the composite mask to a scale of the original version of the image; and
sharpening portions of the enhanced original version that correspond to non-background regions defined by the upscaled composite mask.

### Claim 9 (depends on 1)

The method of claim 1, wherein an original version of the image is a digital photograph of a book page acquired by a camera, the original version downscaled to form the image, the image having a predefined largest dimension.