# US 2021/0342987 A1 · Tone mapping and tone control integrations for image processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342987 A1
- Document ID: US20210342987A1
- Status: Published application
- Kind: A1
- Application number: 17/285,387
- Filed: 2019-06-13
- Published: 2021-11-04
- Earliest claimed: 2018-10-19
- Assignees: GoPro, Inc.
- Applicants: GoPro, Inc.
- Inventors: Guillaume Matthieu Guérin; Bruno César Douady
- Web page: https://patentgrep.com/patent/US20210342987A1

## CPC

- G06T 5/92: based on global image properties
- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- H04N 23/10: for generating image signals from different wavelengths
- H04N 23/51: Housings
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image is processed using a combination of techniques, particularly, in which low frequency information is processed using multiple tone control and high frequency information is processed using local tone mapping. An image is divided into a plurality of blocks including a given block. Low frequency information and high frequency information of the given block are separated. The low frequency information is processed using multiple tone control. The low high frequency information is processed using local tone mapping. A processed image is then produced based on the processed low frequency information and based on the processed high frequency information, the processed image corresponding to the image captured using the image sensor. The processed image is then output for storage or display. Processing the low frequency information can include using a gain curve and bilinear interpolation. Processing the high frequency information can include using an edge preservation filter.

## Claims

### Claim 1

An image capture apparatus, comprising:
an image sensor;
a processor; and
a memory including instructions executable by the processor to:
capture an image using the image sensor;
divide the image into a plurality of blocks including a given block;
separate low frequency information of the given block and high frequency information of the given block;
process the low frequency information of the given block using multiple tone control;
process the high frequency information of the given block using local tone mapping;
produce a processed image based on the processed low frequency information and based on the processed high frequency information, the processed image corresponding to the image captured using the image sensor; and
output the processed image for storage or display.

### Claim 2 (depends on 1)

The image capture apparatus of claim 1, wherein the instructions to process the low frequency information of the given block using multiple tone control include instructions to:
perform multiple tone control against the low frequency information of the given block based on a gain curve computed for the given block and based on results of a bilinear interpolation performed between the given block and one or more blocks adjacent to the given block within the image.

### Claim 3 (depends on 2)

The image capture apparatus of claim 2, wherein the gain curve is a global tone mapping curve.

### Claim 4 (depends on 1)

The image capture apparatus of claim 1, wherein a contrast of the low frequency information within one or more pixels of the given block is enhanced by performing the multiple tone control against the low frequency information of the given block.

### Claim 5 (depends on 1)

The image capture apparatus of claim 1, wherein the instructions to process the high frequency information of the given block using local tone mapping include instructions to:
perform local tone mapping against the high frequency information of the given block using an edge preservation filter.

### Claim 6 (depends on 5)

The image capture apparatus of claim 5, wherein a size of the edge preservation filter is smaller than a size of a filter used to perform the local tone mapping without also performing the multiple tone control.

### Claim 7 (depends on 5)

The image capture apparatus of claim 5, wherein a resolution of the multiple tone control is increased based on a size of the edge preservation filter, wherein increasing the resolution of the multiple tone control compensates for an effect on medium frequency information of the image by the local tone mapping.

### Claim 8 (depends on 1)

The image capture apparatus of claim 1, wherein a contrast of the high frequency information within one or more pixels of the given block is preserved or enhanced by performing the local tone mapping against the high frequency information of the given block.

### Claim 9 (depends on 1)

The image capture apparatus of claim 1, wherein the instructions to process the high frequency information of the given block using local tone mapping include instructions to:
process a pixel of the given block based on a low pass version of a luminance value of the pixel averaged with one or more neighbors of the pixel to produce a processed pixel.

### Claim 10 (depends on 1)

The image capture apparatus of claim 1, wherein the processed image is produced using multi-level lookup tables, wherein a first lookup table of the multi-level lookup tables stores parameter values used to process the low frequency information, wherein one of a plurality of second lookup tables of the multi-level lookup tables stores parameter values used to process the high frequency information.

### Claim 11

An image processor, comprising:
an image capture unit configured to capture an image;
an image processing unit configured to divide the image into blocks and perform image processing against each of the blocks, wherein, for a given block of the blocks, the image processing unit performs the image processing by:
separating low frequency information of the given block and high frequency information of the given block;
performing multiple tone control against the low frequency information of the given block based on a gain curve computed for the given block and based on results of a bilinear interpolation performed between the given block and one or more blocks adjacent to the given block within the image;
performing local tone mapping against the high frequency information of the given block using an edge preservation filter; and
producing a processed block based on results of the multiple tone control and results of the local tone mapping,
wherein a processed image is produced based on the processed block and based on other processed blocks produced by the image processing unit.

### Claim 12 (depends on 11)

The image processor of claim 11, wherein the gain curve is a global tone mapping curve.

### Claim 13 (depends on 11)

The image processor of claim 11, wherein a size of the edge preservation filter is smaller than a size of a filter used to perform the local tone mapping without also performing the multiple tone control.

### Claim 14 (depends on 11)

The image processor of claim 11, wherein performing the multiple tone control against the low frequency information of the given block includes enhancing a contrast of the low frequency information within one or more pixels of the given block.

### Claim 15 (depends on 11)

The image processor of claim 11, wherein performing the local tone mapping against the high frequency information of the given block includes preserving or enhancing a contrast of the high frequency information within one or more pixels of the given block.

### Claim 16 (depends on 11)

The image processor of claim 11, wherein the processed blocks are produced using multi-level lookup tables, wherein a first lookup table of the multi-level lookup tables stores parameter values associated with low frequency information of the image, wherein a plurality of second lookup tables of the multi-level lookup tables each stores parameter values associated with high frequency information of one of the blocks.

### Claim 17 (depends on 11)

The image processor of claim 11, wherein a resolution of the multiple tone control is increased based on a size of the edge preservation filter, wherein increasing the resolution of the multiple tone control compensates for an effect on medium frequency information of the image by the local tone mapping.

### Claim 18

A method, comprising:
filtering a block of an image to separate low frequency information of the block and high frequency information of the block;
processing the low frequency information using multiple tone control and based on a global tone mapping curve;
processing the high frequency information using local tone mapping; and
producing a processed image using the processed low frequency information and the processed high frequency information.

### Claim 19 (depends on 18)

The method of claim 18, wherein processing the low frequency information using the multiple tone control and based on the global tone mapping curve comprises enhancing a contrast of the low frequency information within one or more pixels of the block,
wherein processing the high frequency information using local tone mapping comprises preserving or enhancing a contrast of the high frequency information within the one or more pixels of the block.

### Claim 20 (depends on 18)

The method of claim 18, wherein a size of edge preservation filter used to perform the local tone mapping is smaller than a size of a filter used to perform the local tone mapping without also performing the multiple tone control.
