# US 11,308,589 B2 · Devices, systems, and methods for enhancing images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,589 B2
- Document ID: US11308589B2
- Status: Granted
- Kind: B2
- Application number: 16/400,972
- Filed: 2019-05-01
- Published: 2019-11-07
- Granted: 2022-04-19
- Earliest claimed: 2018-05-03
- Assignees: Canon Virginia, Inc.
- Applicants: Canon Virginia, Inc.
- Inventors: Yunzhe Zhao; Bradley Scott Denney
- Web page: https://patentgrep.com/patent/US11308589B2

## CPC

- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/73: Deblurring; Sharpening
- G06T 5/90: Dynamic range modification of images or parts thereof
- G06T 5/92: based on global image properties
- G06T 2207/20012: Locally adaptive
- G06T 2207/20104: Interactive definition of region of interest [ROI]
- G06T 2207/20192: Edge enhancement; Edge preservation
- G06T 2207/20208: High dynamic range [HDR] image processing
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Devices, systems, and methods obtain an image; perform at least one nonlinear locally-adaptive mapping on the image, thereby generating a revised image; generate an edge-enhanced image based on the revised image; and perform dynamic contrast stretching on the edge-enhanced image, thereby generating an enhanced image.

## Claims

### Claim 1

A method comprising:
obtaining an initial image that is defined in an image space;
performing, in the image space, at least one nonlinear locally-adaptive mapping on the initial image, thereby generating a revised image in the image space, wherein the at least one nonlinear locally-adaptive mapping on the initial image includes a first nonlinear locally-adaptive mapping on the initial image in the image space that generates a first revised value for pixel p in the initial image based on the following: a value of pixel p in the initial image, respective values of pixels in a neighborhood of pixel p in the initial image, a maximum value of respective values of all pixels in the initial image, and a central tendency of the respective values of all the pixels in the initial image;
performing, in the image space, edge enhancement on the initial image, thereby generating an edge-enhanced image in the image space;
generating a synthesized image based on the edge-enhanced image and on the revised image; and
performing, in the image space, dynamic contrast stretching on the revised image in the image space, thereby generating an enhanced image in the image space.

### Claim 2 (depends on 1)

The method of claim 1, wherein performing at least one nonlinear locally-adaptive mapping on the initial image includes:
performing a first nonlinear locally-adaptive mapping on the initial image, thereby generating an interim image in the image space; and
performing a second nonlinear locally-adaptive mapping on the interim image, thereby generating the revised image in the image space.

### Claim 3 (depends on 2)

The method of claim 2, wherein an output pixel value of the first nonlinear locally-adaptive mapping on the initial image can be described by the following:
y
temp
=
(
max
⁡
(
I
i
⁢
⁢
n
)
+
f
+
a
⁢
⁢
1
)
⁢
x
x
+
f
+
a
⁢
⁢
1
,
⁢
where
⁢
⁢
f
=
LP
⁡
(
x
)
+
mean
⁢
⁢
(
I
i
⁢
⁢
n
)
2
,
where ytemp is the output pixel value of the first nonlinear locally-adaptive mapping, where Iin is the initial image, where x is a pixel value in the initial image Iin, where max(Iin) is a maximum pixel value in the initial image Iin, where a1 is a strength-control parameter, where mean(Iin) is a central tendency of all pixel values in the initial image Iin, and where LP(x) is a value at the pixel after the initial image Iin has been convolved with a low-pass filter.

### Claim 4 (depends on 3)

The method of claim 3, wherein an output pixel value of the second nonlinear locally-adaptive mapping on the interim image can be described by the following:
y
out
=
(
max
⁡
(
Y
temp
)
+
h
+
a
⁢
⁢
2
)
⁢
y
temp
y
temp
+
h
+
a
⁢
⁢
2
,
where
⁢
⁢
h
=
LP
⁡
(
y
temp
)
+
mean
⁢
⁢
(
Y
temp
)
2
,
where yout is the output pixel value of the second nonlinear locally-adaptive mapping, where Ktemp is the interim image, where max(Ytemp) is the maximum pixel value in the interim image, where mean(Ytemp) is a central tendency of all pixel values in the interim image, where a2 is a strength-control parameter, and where LP(ytemp) is a value at pixel x after the interim image Ktemp has been convolved with a low-pass filter.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
obtaining an indication of a region of interest in the initial image,
wherein the at least one nonlinear locally-adaptive mapping on the initial image includes a region-of-interest-based nonlinear locally-adaptive mapping.

### Claim 6 (depends on 5)

The method of claim 5, wherein an output pixel value of the region-of-interest-based nonlinear locally-adaptive mapping on the initial image can be described by the following:
y
temp
=
(
max
⁡
(
I
i
⁢
⁢
n
)
+
f
+
a
⁢
⁢
1
)
⁢
x
x
+
f
+
a
⁢
⁢
1
,
where
⁢
⁢
f
=
LP
⁡
(
x
)
+
mean
⁢
⁢
(
I
ROI
)
,
where ytemp is the output pixel value of the region-of-interest-based nonlinear locally-adaptive mapping, where Iin is the initial image, where x is an input pixel value, where max(Iin) is a maximum pixel value in the initial image, where al is a strength-control parameter, where mean(IROI) is a central tendency of all pixel values in the region of interest in the initial image, and where LP(x) is a value at the pixel after the initial image has been convolved with a low-pass filter.

### Claim 7 (depends on 1)

The method of claim 1, wherein the at least one nonlinear locally-adaptive mapping
maps pixel values in the initial image to respective pixel values in the revised image, wherein each pixel value in the initial image maps to a respective pixel value in the revised image; and
maps two pixel values in the initial image that are identical and that have different neighborhood pixel values to two different respective pixel values in the revised image.

### Claim 8 (depends on 7)

The method of claim 7,
wherein a highest pixel value in the initial image is equal to a highest pixel value in the revised image, and wherein a lowest pixel value in the initial image is equal to a lowest pixel value in the revised image.

### Claim 9 (depends on 1)

The method of claim 1, wherein performing the at least one nonlinear locally-adaptive mapping on the initial image includes:
generating a respective scaling factor for each pixel in the initial image, wherein each pixel's respective scaling factor is based, at least in part, on the maximum value of the respective values of all pixels a maximum pixel value in the initial image and on respective pixel values of pixels in a neighborhood of the pixel, and
multiplying each pixel's respective value by the pixel's respective scaling factor.

### Claim 10 (depends on 9)

The method of claim 9, wherein each pixel's respective scaling factor is further based, at least in part, on the pixel's value in the initial image.

### Claim 11

A device comprising:
one or more computer-readable media storing instructions; and
one or more processors that are configured to communicate with the one or more computer-readable media to execute the instructions to cause the device to perform operations comprising:
obtaining an image that is defined in an image space;
performing, in the image space, one or more nonlinear locally-adaptive mappings on the image, thereby generating a revised image in the image space, wherein performing the one or more nonlinear locally-adaptive mappings on the image includes generating a respective scaling factor for each pixel in the image and multiplying each pixel's respective value by the pixel's respective scaling factor;
performing an inverse tone mapping on the image, thereby generating an inverse-tone-mapped image;
generating an adjusted-dynamic-range image in the image space based on the revised image in the image space and on the inverse-tone-mapped image;
synthesizing the revised image and the adjusted-dynamic-range image, thereby generating a synthesized image; and
performing dynamic contrast stretching on the synthesized image.

### Claim 12 (depends on 11)

The device of claim 11, wherein the operations further comprise:
enhancing one or more edges in the image, thereby generating an edge- enhanced image, and
wherein generating the adjusted-dynamic-range image is further based on the edge-enhanced image.

### Claim 13 (depends on 11)

The device of claim 11, wherein the synthesized image includes pixels that have respective values, and
wherein performing the dynamic contrast stretching on the synthesized image includes:
obtaining a first value and a second value, wherein the first value is less than the second value, and
modifying the respective values of the pixels in the synthesized image such that a first percentage of the pixels in the synthesized image have respective values that are lower than the first value and such that a second percentage of the pixels in the synthesized image have respective values that are lower than the second value.

### Claim 14 (depends on 11)

The device of claim 11, wherein the operations further comprise:
obtaining an indication of a region of interest in the image, and
wherein the one or more nonlinear locally-adaptive mappings include at least one region-of-interest-based nonlinear locally-adaptive mapping.

### Claim 15

One or more non-transitory computer-readable media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
obtaining an image that is defined in an image space;
performing, in the image space, a first nonlinear locally-adaptive mapping on the image, thereby generating an interim image in the image space, wherein performing the first nonlinear locally-adaptive mapping on the image includes generating a respective scaling factor for each pixel in the image and multiplying each pixel's respective value by the pixel's respective scaling factor;
performing, in the image space, a second nonlinear locally-adaptive mapping on the interim image in the image space, thereby generating a revised image;
enhancing one or more edges in the image, thereby generating an edge- enhanced image; and
generating a synthesized image based on the edge-enhanced image and on the revised image,
wherein an output pixel value of the first nonlinear locally-adaptive mapping on the image can be described by the following:
y
temp
=
(
max
⁡
(
I
i
⁢
⁢
n
)
+
f
+
a
⁢
⁢
1
)
⁢
x
x
+
f
+
a
⁢
⁢
1
,
⁢
where
⁢
⁢
f
=
LP
⁡
(
x
)
+
mean
⁢
⁢
(
I
i
⁢
⁢
n
)
2
,
where ytemp is the output pixel value of the first nonlinear locally-adaptive mapping, where Iin is the image, where x is a pixel value in the image Iin, where max(Iin) is a maximum pixel value in the image, where a1 is a strength-control parameter, where mean(Iin) is a central tendency of all pixel values in the image, and where LP(x) is a value at the pixel after the image has been convolved with a low-pass filter.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein an output pixel value of the second nonlinear locally-adaptive mapping on the interim image can be described by the following:
y
out
=
(
max
⁡
(
Y
temp
)
+
h
+
a
⁢
⁢
2
)
⁢
y
temp
y
temp
+
h
+
a
⁢
⁢
2
,
where
⁢
⁢
h
=
LP
⁡
(
y
temp
)
+
mean
⁢
⁢
(
Y
temp
)
2
,
where yout is the output pixel value of the second nonlinear locally-adaptive mapping, where ytemp is the interim image, where max(Ytemp) is the maximum pixel value in the interim image, where mean(Ytemp) is a central tendency of all pixel values in the interim image, where a2 is a strength-control parameter, and where LP(ytemp) is a value at pixel x after the interim image Ytemp has been convolved with a low-pass filter.

### Claim 17 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the operations further comprise:
performing dynamic contrast stretching on the synthesized image, thereby generating an adjusted-dynamic-range image.

### Claim 18 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the operations further comprise:
performing an inverse tone mapping on the image, thereby generating an inverse-tone-mapped image, and
generating an adjusted-dynamic-range image based on the inverse-tone-mapped image and on the revised image.