Multiple exposure high dynamic range image capture
AssigneeCSR Technology Inc.
InventorHaim Garten
Life of this patent
Estimating the term…
Abstract
Techniques for creating a High Dynamic Range (HDR) image within a consumer grade digital camera from a series of images of a scene captured at different exposure levels, and displaying the HDR image on the camera's built-in display are provided. The approach employs mixing images of the series to incorporate both scene shadow and highlight details, and the removing of “ghost” image artifacts appearing in the mixed HDR image resulting from movement in the scene over the time the series images are captured. The low computational resource utilization of the image mixing and ghost removal processing operations, along with the ability to commence image mixing and ghost removal prior to the acquisition of all series images, can significantly reduce the time required to generate and display a tone mapped HDR image.
Claims30
A method of mixing a plurality of digital images of a scene, comprising:
- capturing the images at different exposure levels;
- registering counterpart pixels of each image to one another;
- deriving a normalized image exposure level for each image;
- employing the normalized image exposure levels in an image blending process;
- using the image blending process to blend a first selected image and a second selected image to generate an intermediate image, and when the plurality is composed of two images, outputting the intermediate image as a mixed output image;
- and
- when the plurality is composed of more than two images, repeating the image blending process using the previously generated intermediate image in place of the first selected image and another selected image in place of the second selected image until all images have been blended, and outputting the last generated intermediate image as the mixed output image.
A method of removing location shifted replications of scene objects appearing in a mixed image generated by a digital image mixing process applied to a plurality of images acquired at different exposure levels and different times, with the images having their counterpart pixels registered to one another, comprising:
- normalizing luma values of the images to a specific standard deviation and mean;
- detecting local motion between at least one reference image and at least one comparison image;
- clustering comparison image pixels having local motion into patches;
- selecting corresponding patches from the reference image;
- generating a binary connected image by logically ORing the patches generated from particular reference images together;
- and
- blending the mixed image with the reference images, with each reference image weighted by a weighting value computed from the binary connected image, to produce an output image.
An image capturing device that captures a plurality of digital images of a scene at different exposure levels, comprising:
- an image registration processor that registers counterpart pixels of each image of the plurality to one another;
- an image mixer that combines multiple images of the plurality to generate a single image, wherein the image mixer comprises:
- an image normalizer that normalizes the image exposure level for each image;
- and
- an image blender that uses the normalized exposure level to blend a first selected image and a second selected image to generate an intermediate image, and when the plurality is composed of two images, outputs the intermediate image as a mixed output image, and
- when the plurality is composed of more than two images, repeatedly blends the previously generated intermediate image in place of the first selected image and another selected image in place of the second selected image until all images have been blended, and outputs the last generated intermediate image as the mixed output image.
An image capturing device that captures a plurality of digital images of a scene at different exposure levels and different times, comprising:
- an image registration processor that registers the counterpart pixels of each image to one another;
- an image mixer that combines multiple images to generate a mixed image;
- and
- a ghost remover that removes location shifted replications of scene objects appearing in the mixed image, where the ghost remover comprises:
- an image normalizer that normalizes the image exposure level for each image to a specific standard deviation and mean;
- a reference image selection circuit that selects at least one reference image from the images;
- a local motion detector circuit that detects local motion between the reference image and at least one comparison image;
- a clustering circuit that clusters comparison image pixels having local motion into patches and selects corresponding patches from the reference image;
- a binary connected image generator circuit that logically ORs the patches generated from particular reference images together,
- wherein the image mixer combines the mixed image with the reference images in a final stage of processing, with each reference image weighted by a weighting value computed from the binary connected image, to produce an output image.
A digital camera that captures a plurality of digital images of a scene at different exposure levels at different times, and generates therefrom a tone mapped high dynamic range image, comprising:
- an image registration processor that registers counterpart pixels of each image of the plurality to one another;
- an image mixer that combines multiple images of the plurality to generate a single image, wherein the image mixer comprises:
- an image normalizer that normalizes the image exposure level for each image;
- and
- an image blender that uses the normalized exposure level to blend a first selected image and a second selected image to generate an intermediate image, and
- when the plurality is composed of two images, outputs the intermediate image as a mixed image, and
- when the plurality is composed of more than two images, repeatedly blends the previously generated intermediate image in place of the first selected image and another selected image in place of the second selected image until all images have been blended, and outputs the last generated intermediate image as the mixed image;
- a ghost remover that removes location shifted replications of scene objects appearing in the mixed image, where the ghost remover comprises:
- an image normalizer that normalizes the image exposure level for each image to a specific standard deviation and mean;
- a reference image selection circuit that selects at least one reference image from the images;
- a local motion detector circuit that detects local motion between the reference image and at least one comparison image;
- a clustering circuit that clusters comparison image pixels having local motion into patches and selects corresponding patches from the reference image;
- a binary connected image generator circuit that logically ORs the patches generated from particular reference images together,
- wherein the image mixer combines the mixed image with the reference images in a final stage of processing, with each reference image weighted by a weighting value computed from the binary connected image, to produce a processed mixed image;
- and
- a tone mapping processor that maps the processed mixed image pixels to display pixels with the number of digital bits that can be presented on a built-in digital camera image display.
Dependencies are inferred from the claim text.