# US 8,570,389 B2 · Enhancing digital photography > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,570,389 B2 - Document ID: US8570389B2 - Status: Granted - Kind: B2 - Application number: 11/187,355 - Filed: 2005-07-21 - Published: 2006-01-26 - Granted: 2013-10-29 - Earliest claimed: 2004-07-22 - Assignees: Broadcom Corporation - Applicants: Noam Sorek; Ilya Vitsnudel - Inventors: Noam Sorek; Ilya Vitsnudel - Web page: https://patentgrep.com/patent/US8570389B2 ## CPC - H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors - H04N 23/70: Circuitry for compensating brightness variation in the scene - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method for enhancing imaging in low light conditions. The method comprises acquiring two or more images, long images relating to long exposure times and a short images relating to short exposure times; processing the images to separate zones with high image details from zones of low image details; and reconstructing a final image where image information is selected from the short image in zones with high image details and from the long image in zones with low image details. ## Claims ### Claim 1 A method for enhancing imaging in low light conditions, the method comprising: acquiring a sequence of two or more images comprising at least a long image relating to long exposure time and a short image relating to short exposure time; processing the sequence of two or more images by: identifying at least one image zone of the sequence of two or more images having corresponding image levels between the long image and the short image upon gray level alignment; filtering the short image and the long image to identify image edges in the short image and the long image generating a filtered short image and a filtered long image; based on the identified image edges, masking the filtered short image and the filtered long image to identify high activity regions of the short image and the long image; based on the masking, identifying high resolution areas of the filtered short image and generating an image combination mask; and reconstructing a final image by combining the long image and the short image with the image combination mask. ### Claim 2 (depends on 1) The method according to claim 1, wherein the gray level alignment comprises estimating an offset and a ratio between the long image and the short image to align the sequence of two or more images. ### Claim 3 (depends on 2) The method according to claim 2, further comprising using the estimated offset and ratio to correct the short image to have proper gray level alignment with the long image. ### Claim 4 (depends on 1) The method according to claim 1, further comprising: performing a geometrical registration process to align the short image and the long image; and when the geometrical registration process indicates excessive motion common to the sequence of two or more images, discarding the long image and producing the final image from the filtered short image. ### Claim 5 (depends on 4) The method according to claim 4, wherein, when the geometrical registration process indicates a presence of global motion between the filtered short image and the filtered long image, masking comprises discarding the long image and producing the final image from the filtered short image at selected image margins of the short image. ### Claim 6 (depends on 4) The method according to claim 4, wherein performing the geometrical registration process comprises estimating a cross-correlation between the filtered short image and the filtered long image in a vertical direction and a horizontal direction. ### Claim 7 (depends on 6) The method according to claim 6, wherein the cross-correlation is performed along a plurality of lines among the filtered short image and the filtered long image. ### Claim 8 (depends on 1) The method according to claim 1, further comprising applying a smoothing filter to the image combination mask. ### Claim 9 (depends on 8) The method according to claim 8, wherein the smoothing filter is an infinite impulse response (IIR) filter. ### Claim 10 (depends on 9) The method according to claim 9, wherein the smoothing filter has a predefined pole. ### Claim 11 An image processing device comprising: a sensor; a memory; a processor in communication with the memory and the sensor, the processor being operable to: acquire a sequence of two or more images comprising a plurality of image zones from the sensor, the sequence of two or more images comprising at least a long image relating to a long exposure time and a short image relating to a short exposure time; identify at least one image zone of the sequence of two or more images having corresponding image levels between the long image and the short image upon gray level alignment; filter the short image and the long image to identify image edges in the short image and the long image generating a filtered short image and a filtered long image; based on the identified image edges, mask the filtered short image and the filtered long image to identify frequency content above a predetermined threshold; based on the masking, identify high resolution areas of the filtered short image and generate an image combination mask; compare the filtered short image and the filtered long image to identify regions of local motion; and reconstruct a final image with the image combination mask, wherein image information is selected from the short image in said at least one identified image zones with frequency content above the predetermined threshold and from the long image in other zones. ### Claim 12 (depends on 11) The image processing device according to claim 11, wherein the gray level alignment comprises estimating an offset and ratio between the long image and the short image to align the sequence of two or more images. ### Claim 13 (depends on 12) The image processing device according to claim 12, further comprising using the estimated offset and ratio to correct the short image to have proper gray level alignment with the long image. ### Claim 14 (depends on 11) The image processing device according to claim 11, wherein the processor is further operable to: perform a geometrical registration process to align the short image and the long image; and when the geometrical registration process indicates excessive motion common to the sequence of two or more images, discard the long image and producing the final image from the filtered short image. ### Claim 15 (depends on 14) The image processing device according to claim 14, wherein, when the processor determines that the geometrical registration process indicates presence of global motion between the short image and the long image, the processor is further operable to discard the long image and produce the final image from the filtered short image. ### Claim 16 (depends on 14) The image processing device according to claim 14, further comprising a detector that detects motion of the image processing device relative to a subject of an image of the sequence of two or more images to detect the excessive motion common to the sequence of two or more images. ### Claim 17 (depends on 14) The image processing device according to claim 14, wherein the processor is operable to perform the geometrical registration process comprising estimating a cross-correlation between the short image and the long image in a vertical direction and a horizontal direction. ### Claim 18 (depends on 17) The image processing device according to claim 17, wherein the cross-correlation comprises, in the processor, cross-correlating along a plurality of lines among the short image and the long image. ### Claim 19 (depends on 11) The image processing device according to claim 11, wherein the image processing device comprises a camera. ### Claim 20 (depends on 11) The image processing device according to claim 11, further comprising a gyro configured to assess motion of the image processing device.