# US 9,883,125 B2 · Imaging systems and methods for generating motion-compensated high-dynamic-range images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,883,125 B2 - Document ID: US9883125B2 - Status: Granted - Kind: B2 - Application number: 14/563,887 - Filed: 2014-12-08 - Published: 2015-04-02 - Granted: 2018-01-30 - Earliest claimed: 2011-10-06 - Assignees: Semiconductor Components Industries, LLC - Applicants: Semiconductor Components Industries, LLC - Inventors: Huai Dong Li; Peng Lin - Web page: https://patentgrep.com/patent/US9883125B2 ## CPC - H04N 25/589: with different integration times, e.g. short and long exposures - H04N 23/6811: {based on the image signal} - H04N 23/73: by influencing the exposure time - H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors - H04N 25/58: involving two or more exposures - H04N 25/583: with different integration times - H04N 25/585: with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Electronic devices may include image sensors. Image sensors may be used to capture images having rows of long-exposure image pixel values that are interleaved with rows of short-exposure image pixel values. The long-exposure and short-exposure values in each interleaved image frame may be interpolated to form interpolated values. A combined long-exposure image and a combined short-exposure image may be generated using the long-exposure and the short-exposure values from the interleaved image frames and the interpolated values from a selected one of the interleaved image frames. The combined long-exposure and short-exposure images may each include image pixel values from either of the interleaved image frames in a non-motion edge region and image pixel values based only on the image pixel values or the interpolated values from the selected one of the interleaved images in a motion or non-edge region. High-dynamic-range images may be generated using the combined long-exposure and short-exposure images. ## Claims ### Claim 1 A method of capturing images, comprising: capturing a plurality of successive images, wherein a first one of the plurality of successive images has short-exposure image pixel values and long-exposure image pixel values in a first arrangement, and wherein a second one of the plurality of successive images has short-exposure image pixel values and long-exposure image pixel values in a second arrangement that is different than the first arrangement, wherein the short-exposure image pixel values and the long-exposure image pixel values in the first one of the plurality of successive images are captured by first and second respective sets of pixels, wherein the first set of pixels captures the long-exposure image pixel values in the second one of the plurality of successive images and the second set of pixels captures the short-exposure image pixel values in the second one of the plurality of successive images; and processing the first one of the plurality of successive images and the second one of the plurality of successive images to generate a motion-compensated high dynamic range image. ### Claim 2 (depends on 1) The method defined in claim 1, wherein processing the plurality of successive images further comprises: with the first one of the plurality of successive images, generating at least a first interpolated image based on a selected one of the short-exposure image pixel values and the long-exposure image pixel values of the first image. ### Claim 3 (depends on 2) The method defined in claim 2, wherein processing the plurality of successive images further comprises: with the second one of the plurality of successive images, generating at least a second interpolated image based on a selected one of the short-exposure image pixel values and the long-exposure image pixel values of the second image. ### Claim 4 (depends on 3) The method defined in claim 3, wherein processing the plurality of successive images further comprises: generating a motion map based on the at least first interpolated image and the at least second interpolated image. ### Claim 5 (depends on 1) The method defined in claim 1, wherein processing the plurality of successive images comprises: generating an interpolated short-exposure image based on the short-exposure image pixel values of the first one of the plurality of successive images; and generating an interpolated long-exposure image based on the long-exposure image pixel values of the first one of the plurality of successive images. ### Claim 6 (depends on 5) The method defined in claim 5, wherein processing the plurality of successive images further comprises: with the interpolated short-exposure image and the interpolated long-exposure image, generating an edge map. ### Claim 7 (depends on 5) The method defined in claim 5, wherein processing the plurality of successive images further comprises: with the first one of the plurality of successive images, the second one of the plurality of successive images, and the interpolated short-exposure image, generating a short-exposure image; and with the first one of the plurality of successive images, the second one of the plurality of successive images, and the interpolated long-exposure image, generating a long-exposure image. ### Claim 8 (depends on 7) The method defined in claim 7, wherein processing the plurality of successive images further comprises: generating a reconstruction map; and using the reconstruction map to generate the short-exposure image and the long-exposure image. ### Claim 9 (depends on 8) The method defined in claim 8, wherein processing the plurality of successive images further comprises: generating a motion map based on the first image and the second image; generating a first edge map based on the first image; generating a second edge map based on the second image; and with the motion map, the first edge map, and the second edge map, generating the reconstruction map. ### Claim 10 (depends on 7) The method defined in claim 7, wherein processing the plurality of successive images further comprises: with the short-exposure image and the long-exposure image, generating the motion-compensated high dynamic range image. ### Claim 11 (depends on 1) The method defined in claim 1, wherein the first one of the plurality of successive images has intermediate-exposure image pixel values in the first arrangement, and wherein the second one of the plurality of successive images has intermediate-exposure pixel values in the second arrangement. ### Claim 12 A method of generating a motion-compensated high dynamic range image using processing circuitry and an image sensor having pixels, the method comprising: with the image sensor, capturing a first image that includes a first set of pixels having image pixel values for a first exposure time and a second set of pixels having image pixel values for a second exposure time that is different than the first exposure time; with the image sensor, capturing a second image that includes the first set of pixels having image pixel values for the second exposure time and the second set of pixels having image pixel values for the first exposure time; and with the processing circuitry, generating the motion-compensated high dynamic range image by processing the first and second images. ### Claim 13 (depends on 12) The method defined in claim 12, further comprising: selecting one of the first and second images as a reference image. ### Claim 14 (depends on 13) The method defined in claim 13, further comprising: identifying a motion region based on the first image and the second image. ### Claim 15 (depends on 14) The method defined in claim 14, further comprising: identifying at least one edge region in the first image; and identifying at least one edge region in the second image. ### Claim 16 (depends on 15) The method defined in claim 15, wherein the reference image is selected based on the motion region, the at least one edge region in the first image, and the at least one edge region in the second image. ### Claim 17 (depends on 13) The method defined in claim 13, wherein the motion-compensated high dynamic range image is generated from the reference image. ### Claim 18 (depends on 13) The method defined in claim 13, wherein capturing the first image comprises capturing the first image that includes a third set of pixels having image pixel values for a third exposure time that is different than the first and second exposure times. ### Claim 19 A system, comprising: a central processing unit; memory; input-output circuitry; an imaging device, wherein the imaging device comprises: an image sensor configured to capture a first image and a second image, wherein the first image is captured using a first set of pixels to generate first short-exposure pixel values and a second set of pixels to capture first long-exposure pixel values, and wherein the second image is captured using the first set of pixels to generate second long-exposure pixel values and the second set of pixels to capture second short-exposure pixel values; and processing circuitry configured to process the first and second images to generate a motion-compensated high dynamic range output image. ### Claim 20 (depends on 19) The system of claim 19, wherein the first image is further captured using a third set of pixels to generate third short-exposure pixel values, wherein the second image is further captured using the third set of pixels to generate third long-exposure pixel values, and wherein the processing circuitry is further configured to generate interpolated short-exposure pixel values for the second set of pixels based on the first and third short-exposure pixel values and to generate interpolated long-exposure pixel values based on the second and third long-exposure pixel values. ### Claim 21 (depends on 20) The system of claim 20, wherein the first image is further captured using a fourth set of pixels to generate fourth long-exposure pixel values, wherein the second image is further captured using the fourth set of pixels to generate fourth short-exposure pixel values, and wherein the processing circuitry is further configured to generate additional interpolated long-exposure pixel values for the third set of pixels based on the first and fourth long-exposure pixel values and to generate additional interpolated short-exposure pixel values based on the second and fourth short-exposure pixel values. ### Claim 22 (depends on 21) The system of claim 21, wherein the processing circuitry generates the motion-compensated high dynamic range output image by selecting one of the interpolated long-exposure pixel values and the additional interpolated long-exposure pixel values and by selecting one of the interpolated short-exposure pixel values and the additional interpolated short-exposure pixel values. ### Claim 23 (depends on 22) The system of claim 22, wherein the processing circuitry generates the motion-compensated high dynamic range output image by combining the selected one of the interpolated long-exposure pixel values and the additional interpolated long exposure pixel values, the selected one of the interpolated short-exposure pixel values and the additional interpolated short-exposure pixel values, the first image, and the second image.