US 9,852,499 B2Granted

Automatic selection of optimum algorithms for high dynamic range image processing based on scene classification

AssigneeKonica Minolta Laboratory U.S.A., Inc.

InventorsWei Ming, Xiaonong Zhan

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method for processing high dynamic range (HDR) images by selecting preferred tone mapping operators and gamut mapping algorithms based on scene classification. Scenes are classified into indoor scenes, outdoor scenes, and scenes with people, and tone mapping operators and gamut mapping algorithms are selected on that basis. Prior to scene classification, the multiple images taken at various exposure values are fused into a low dynamic range (LDR) image using an exposure fusing algorithm, and scene classification is performed using the fused LDR image. Then, the HDR image generated from the multiple images are tone mapped into a LDR image using the selected tone mapping operator and then gamut mapped to the color space of the output device such as printer.

Claims13

Dependent claims

A method implemented in a data processing apparatus for processing a set of multiple input images of a scene taken at various exposure values, comprising:

(a) using one or more of the multiple input images, classifying the scene into one of a plurality of scene classes, wherein the plurality of scene classes either include a class of scenes that contains a significant presence of people, a class of outdoor scenes and a class of indoor scenes, or include a class of scenes that contains a significant presence of people, a class of daylight scenes and a class of night scenes,
wherein step (a) comprises:
(a1) selecting a subset of two or more images from the multiple input images;
(a2) down-sampling the selected images;
(a3) after down-sampling, fusing the selected images into a single fused image, by generating a weight map for each selected image and combining the selected images using the weight maps, the weight maps being generated based on only one of saturation, contrast, and well-exposedness of each pixel in each selected image;
and
(a4) classifying the fused image into one of the plurality of scene classes;
(b) based on the scene class determined in step (a), selecting one of a plurality of pre-stored tone mapping operators;
(c) merging the multiple input images to generate a high dynamic range (HDR) image;
(d) tone mapping the HDR image using the tone mapping operator selected in step (b) to generate a low dynamic range (LDR) image;
(e) based on the scene class determined in step (a), selecting one of a plurality of pre-stored gamut mapping algorithms;
and
(f) using the gamut mapping algorithm selected in step (e), converting the LDR image generated in step (d) from a color space of the image to a color space of an output device.

10 more elements

A computer program product comprising a computer usable non-transitory medium having a computer readable program code embedded therein for controlling a data processing apparatus, the computer readable program code configured to cause the data processing apparatus to execute a process for processing a set of multiple input images of a scene taken at various exposure values, the process comprising:

(a) using one or more of the multiple input images, classifying the scene into one of a plurality of scene classes, wherein the plurality of scene classes either include a class of scenes that contains a significant presence of people, a class of outdoor scenes and a class of indoor scenes, or include a class of scenes that contains a significant presence of people, a class of daylight scenes and a class of night scenes,
wherein step (a) comprises:
(a1) selecting a subset of two or more images from the multiple input images;
(a2) down-sampling the selected images;
(a3) after down-sampling, fusing the selected images into a single fused image, by generating a weight map for each selected image and combining the selected images using the weight maps, the weight maps being generated based on only one of saturation, contrast, and well-exposedness of each pixel in each selected image;
and
(a4) classifying the fused image into one of the plurality of scene classes;
(b) based on the scene class determined in step (a), selecting one of a plurality of pre-stored tone mapping operators;
(c) merging the multiple input images to generate a high dynamic range (HDR) image;
(d) tone mapping the HDR image using the tone mapping operator selected in step (b) to generate a low dynamic range (LDR) image;
(e) based on the scene class determined in step (a), selecting one of a plurality of pre-stored gamut mapping algorithms;
and
(f) using the gamut mapping algorithm selected in step (e), converting the LDR image generated in step (d) from a color space of the image to a color space of an output device.

10 more elements

Dependencies are inferred from the claim text.

Facts

Application: 14/105,652Filed 13 Dec 2013
Published: US 2015/0170389 A118 Jun 2015
Granted: 26 Dec 2017
Term ends: Not estimatedThe source records don’t support an estimate.

Classification11 codes

G06T 5/50
Image enhancement or restoration: using two or more images, e.g. averaging or subtraction
G06T 5/92
Dynamic range modification of images or parts thereof: based on global image properties
G06V 20/35
Scenes; Scene-specific elements: categorising the entire scene, e.g. birthday party or wedding scene
G06T 2207/10016
Image acquisition modality: video; Image sequence