# US 7,609,320 B2 · Imaging device, imaging device image output method, and computer for assigning a dynamic range > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,609,320 B2 - Document ID: US7609320B2 - Status: Granted - Kind: B2 - Application number: 10/528,968 - Filed: 2003-09-18 - Published: 2006-02-16 - Granted: 2009-10-27 - Earliest claimed: 2002-09-25 - Assignees: Sony Corporation - Inventors: Keisuke Okamura - Web page: https://patentgrep.com/patent/US7609320B2 ## CPC - H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors - H04N 23/70: Circuitry for compensating brightness variation in the scene - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image pickup apparatus for performing dynamic assignment of an output image in a dynamic range according to a subject. An image pickup device picks up an image of a subject, a signal processing section generates a composite image having a relatively wider dynamic range than at least either of the dynamic ranges of a long-time exposure image picked up with a relatively long exposure time by the image pickup device and a short-time exposure image picked up with a relatively short exposure time by the image pickup device, by synthesizing the long-time exposure image and the short-time exposure image A, control section compresses the composite image and dynamically varies the assignment proportion of a high luminance dynamic range to a low-middle luminance dynamic range in a dynamic range of an output image to be outputted as a video signal. ## Claims ### Claim 1 An image pickup apparatus comprising: an image pickup device for picking up an image of a subject; a signal processing section for generating a composite image, the composite image having a relatively wider dynamic range than at least either a dynamic range of a long-time exposure image picked up with a relatively long exposure time by said image pickup device or a dynamic range of a short-time exposure image picked up with a relatively short exposure time by said image pickup device, by synthesizing said long-time exposure image and said short-time exposure image; and a control section for compressing said composite image and dynamically varying an assignment proportion of a high luminance dynamic range to a low-middle luminance dynamic range in a dynamic range of an output image to be outputted as a video signal, wherein the signal processing section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same. ### Claim 2 (depends on 1) The image pickup apparatus according to claim 1, wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least a luminance region which occupies said composite image. ### Claim 3 (depends on 1) The image pickup apparatus according to claim 1, wherein said control section corrects said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range each time said composite image is generated. ### Claim 4 (depends on 2) The image pickup apparatus according to claim 2, wherein said luminance region is at least either a high luminance region or a low-middle luminance region. ### Claim 5 (depends on 1) The image pickup apparatus according to claim 1, wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least an average luminance signal level of a high luminance region which occupies said composite image. ### Claim 6 (depends on 1) The image pickup apparatus according to claim 1, wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least an average luminance signal level of a low-middle luminance region which occupies said composite image. ### Claim 7 (depends on 1) The image pickup apparatus according to claim 1, wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least a high luminance region which occupies said composite image. ### Claim 8 (depends on 1) The image pickup apparatus according to claim 1, wherein said control section dynamically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range according to at least a low middle luminance region which occupies said composite image. ### Claim 9 (depends on 1) The image pickup apparatus according to claim 1, wherein said control section at least monotonically varies said assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range. ### Claim 10 An image pickup apparatus comprising: an image pickup device for picking up an image of a subject; a signal processing section for generating a composite image, the composite image having a relatively wider dynamic range than at least either a dynamic range of a long-time exposure image picked up with a relatively long exposure time by said image pickup device or a dynamic range of a short-time exposure image picked up with a relatively short exposure time by said image pickup device, by synthesizing said long-time exposure image and said short-time exposure image; and a control section for compressing said composite image and dynamically assigning the dynamic range of said composite image to the dynamic range of an output image to be outputted as a video signal, wherein the signal processing section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same. ### Claim 11 (depends on 10) The image pickup apparatus according to claim 10, wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image according to at least a luminance region which occupies said composite image. ### Claim 12 (depends on 10) The image pickup apparatus according to claim 10, wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image each time said composite image is generated. ### Claim 13 (depends on 10) The image pickup apparatus according to claim 10, wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image according to at least a high luminance region which occupies said composite image. ### Claim 14 (depends on 13) The image pickup apparatus according to claim 13, wherein said control section dynamically assigns the dynamic range of said composite image to the dynamic range of said output image according to at least an average luminance signal level of said high luminance region which occupies said composite image. ### Claim 15 (depends on 10) The image pickup apparatus according to claim 10, wherein said dynamic range is at least either a high luminance dynamic range or a low-middle luminance dynamic range. ### Claim 16 (depends on 15) The image pickup apparatus according to claim 15, wherein said control section dynamically assigns assignment of at least either said high luminance dynamic range or said low-middle luminance dynamic range of said output image according to a high luminance region which occupies said composite image. ### Claim 17 (depends on 15) The image pickup apparatus according to claim 15, wherein said control section dynamically assigns a section of said high luminance dynamic range of said output image to said low-middle luminance dynamic range according to at least a decrease of a high luminance region which occupies said composite image. ### Claim 18 (depends on 15) The image pickup apparatus according to claim 15, wherein said control section dynamically assigns a section of said high luminance dynamic range of said output image to said low-middle luminance dynamic range according to at least a decrease of an average luminance signal level of a high luminance region which occupies said composite image. ### Claim 19 (depends on 15) The image pickup apparatus according to claim 15, wherein said control section dynamically assigns a section of said low-middle luminance dynamic range of said output image to said high luminance dynamic range according to at least an increase of a high luminance region which occupies said composite image. ### Claim 20 (depends on 15) The image pickup apparatus according to claim 15, wherein said control section dynamically assigns a section of said low-middle luminance dynamic range of said output image to said high luminance dynamic range according to at least an increase of an average luminance signal level of a high luminance region which occupies said composite image. ### Claim 21 (depends on 15) The image pickup apparatus according to claim 15, wherein said control section at least monotonically varies an assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range of the said output image. ### Claim 22 An image pickup apparatus comprising: an image pickup device for picking up an image of a subject; a detection section for detecting an image signal of a long-time exposure image picked up with a relatively long exposure time by said image pickup device, and an image signal of a short-time exposure image picked up with a relatively short exposure time by said image pickup device; a synthesis section for generating a composite image from said long-time exposure image and said short-time exposure image on the basis of a switch luminance signal level determined from said image signals; a control section for compressing said composite image according to a luminance region which occupies said composite image, and dynamically assigning a dynamic range of an output image to be outputted as a video signal; and a compression section for compressing the dynamic range of said composite image on the basis of dynamic assignment of said dynamic range of said output image, wherein the detection section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same. ### Claim 23 (depends on 22) The image pickup apparatus according to claim 22, wherein said luminance region is at either a high luminance region or a low-middle luminance region. ### Claim 24 (depends on 22) The image pickup apparatus according to claim 22, wherein said synthesis section acquires, from said short-time exposure image, pixels corresponding to at least a higher luminance signal level than said switch luminance signal level among pixels constructed in said composite image. ### Claim 25 (depends on 22) The image pickup apparatus according to claim 22 wherein said synthesis section acquires, from said long-time exposure image, pixels corresponding to at least a lower luminance signal level than said switch luminance signal level among pixels constructed in said composite image. ### Claim 26 (depends on 22) The image pickup apparatus according to claim 22, wherein said dynamic range is at least either a high luminance dynamic range or a low-middle luminance dynamic range. ### Claim 27 (depends on 22) The image pickup apparatus according to claim 22, wherein said control section determines a compression gain for compressing a luminance signal level of said composite image on the basis of at least an assignment proportion of a high luminance dynamic range of said output image to a low-middle luminance dynamic range thereof ### Claim 28 (depends on 22) The image pickup apparatus according to claim 22, wherein said control section determines at least a high luminance compression gain for compressing a luminance signal level of a high luminance region in said composite image and a low-middle luminance compression gain for compressing a luminance signal level of a low-middle luminance region. ### Claim 29 (depends on 28) The image pickup apparatus according to claim 28, wherein said control section further includes a compression gain calculation section for determining, for each luminance signal level of said composite image, at least either a final high luminance compression gain or a final low-middle luminance compression gain which are to be used by said compression section, on the basis of at least either said high luminance compression gain or said low-middle luminance compression gain. ### Claim 30 (depends on 22) The image pickup apparatus according to claim 22, wherein said control section dynamically varies an assignment proportion between a high luminance dynamic range and a low-middle luminance dynamic range of said output image according to at least a high luminance region which occupies said composite image. ### Claim 31 (depends on 30) The image pickup apparatus according to claim 30, wherein said control section at least monotonically varies the assignment proportion of said high luminance dynamic range to said low-middle luminance dynamic range of said output image. ### Claim 32 An image pickup apparatus comprising: an image pickup device for picking up an image of a subject; a detection section for detecting an image signal of a long-time exposure image picked up with a relatively long exposure time by said image pickup device, and an image signal of a short-time exposure image picked up with a relatively short exposure time by said image pickup device; a synthesis section for excluding said luminance signal level from a target to be synthesized, when at least said image signal corresponding to a luminance signal level of either said long-time exposure image or said short-time exposure image is absent, and synthesizing said long-time exposure image and said short-time exposure image on the basis of a switch luminance signal level; a control section which dynamically assigns a dynamic range of an output image to be outputted as a video signal in which a composite image is compressed, according to a luminance region which occupies said composite image in which said long-time exposure image and said short-time exposure image are synthesized; and a compression section for compressing the dynamic range of said composite image on the basis of dynamic assignment of said dynamic range of said output image, wherein the detection section determines a black level based on a periodicity of image signals that include alternatively disposed long-time exposure image signal and short-time exposure image signal, and wherein an exposure ratio of the relatively long exposure time to the relatively short exposure time is multiplied by the short-time exposure image so that an amount of input light of the long-time exposure image and the short-time exposure image is substantially the same. ### Claim 33 (depends on 32) The image pickup apparatus according to claim 32, wherein said synthesis section selects said luminance signal level lower than said switch luminance signal level in said long-time exposure image, as a target for said composite image. ### Claim 34 (depends on 32) The image pickup apparatus according to claim 32, wherein said synthesis section selects said luminance signal level higher than said switch luminance signal level in said short-time exposure image, as a target for said composite image. ### Claim 35 (depends on 32) The image pickup apparatus according to claim 32, wherein said luminance region is at least either a high luminance region or a low-middle luminance region. ### Claim 36 (depends on 32) The image pickup apparatus according to claim 32, wherein said dynamic range is at least either a high luminance dynamic range or a low-middle luminance dynamic range. ### Claim 37 (depends on 32) The image pickup apparatus according to claim 32, wherein said control section dynamically varies an assignment proportion of a high luminance dynamic range to a low-middle luminance dynamic range of said output image according to at least a high luminance region which occupies said composite image.