# US 9,438,810 B2 · Dual sensor camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,810 B2 - Document ID: US9438810B2 - Status: Granted - Kind: B2 - Application number: 14/834,035 - Filed: 2015-08-24 - Published: 2015-12-17 - Granted: 2016-09-06 - Earliest claimed: 2009-03-19 - Assignees: DigitalOptics Corporation - Applicants: DigitalOptics Corporation - Inventors: John D. Griffith; Jisoo Lee - Web page: https://patentgrep.com/patent/US9438810B2 ## CPC - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - G06T 5/50: using two or more images, e.g. averaging or subtraction - H04N 5/2628: {Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation} - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils - H04N 23/60: Control of cameras or camera modules - H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming - G02B 27/0075: {with means for altering, e.g. increasing, the depth of field or depth of focus} - G06T 5/73: Deblurring; Sharpening - G06T 2207/20221: Image fusion; Image merging - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract A dual sensor camera that uses two aligned sensors each having a separate lens of different focal length but the same f-number. The wider FOV image from one sensor is combined with the narrower FOV image from the other sensor to form a combined image. Up-sampling of the wide FOV image and down-sampling of the narrow FOV image is performed. The longer focal length lens may have certain aberrations introduced so that Extended Depth of Field (EDoF) processing can be used to give the narrow FOV image approximately the same depth of field as the wide FOV image so that a noticeable difference in depth of field is not see in the combined image. ## Claims ### Claim 1 A method of generating a combined image with a camera, the method comprising: receiving light with a first sensor from a first lens having a first focal length; capturing a first image formed on the first sensor with the first sensor; receiving light with a second sensor from a second lens having a second focal length that is longer than the first focal length; capturing a second image formed on the second sensor with the second sensor, the combination of the first sensor and the first lens and the combination of the second sensor and the second lens being directed toward the same subject; and combining the first image and the second image together to form a single combined image with the second image forming a central portion of the single combined image and a peripheral portion of the first image forming a peripheral portion of the single combined image; and wherein the step of combining the first and the second images includes at least one of up-sampling the first image and down-sampling the second image; and when the first image is up-sampled, the first image is sharpened after the up-sampling. ### Claim 2 (depends on 1) The method of claim 1, wherein the up-sampling is performed with a scaling factor A and the down-sampling is performed with a scaling factor B. ### Claim 3 (depends on 2) The method of claim 2, wherein: Z is the ratio of the field of view (FOV) of the first sensor to the FOV of the second sensor; and Z=A×B. ### Claim 4 (depends on 3) The method of claim 3, wherein A is a value in the range between and including 1 and Z. ### Claim 5 (depends on 2) The method of claim 2, wherein: A is a value in the range between and including 1 and Z; and Z is the ratio of the field of view (FOV) of the first sensor to the FOV of the second sensor. ### Claim 6 (depends on 1) The method of claim 1, further comprising applying a mask to the first image before the first image is combined with the second image, the mask blocking a central portion of the first image and allowing the peripheral portion of the first image to be used in forming the single combined image. ### Claim 7 (depends on 1) The method of claim 1, further comprising applying a mask to the second image before the second image is combined with the first image, the mask blocking a peripheral portion of the second image and allowing a central portion of the second image to be used in forming the single combined image. ### Claim 8 (depends on 1) The method of claim 1, wherein the first lens and the second lens have substantially equal f-numbers. ### Claim 9 (depends on 1) The method of claim 1, wherein the first lens and the second lens are selected such that the illuminance on the first sensor and the illuminance on the second sensor are equivalent. ### Claim 10 (depends on 1) The method of claim 1, wherein the amount of up-sampling is determined based at least in part on a digital zoom setting selected by a user. ### Claim 11 (depends on 1) The method of claim 1, wherein the central portion and the peripheral portion of the single combined image have similar levels of at least one of brightness, background noise, motion artifacts, and depth of field. ### Claim 12 (depends on 1) The method of claim 1, wherein: the second lens has been designed to work with extended depth of field (EDoF) processing by the introduction of specified aberrations into the second image formed on the second sensor; and the second image is subjected to EDoF processing to focus the second image. ### Claim 13 (depends on 12) The method of claim 12, wherein: the first image has a first depth of field determined by the combination of the first sensor and the first lens; and the EDoF processing of the second image results in the second image having a second depth of field that is substantially equal to the first depth of field. ### Claim 14 (depends on 12) The method of claim 12, wherein: the first lens has been designed to work with extended depth of field (EDoF) processing by the introduction of specified aberrations into the first image formed on the first sensor; and the first image is subjected to EDoF processing to focus the first image. ### Claim 15 A method of generating a combined image with a camera, the method comprising: receiving light with a first sensor from a first lens having a first focal length; capturing a first image formed on the first sensor with the first sensor; receiving light with a second sensor from a second lens having a second focal length that is longer than the first focal length; capturing a second image formed on the second sensor with the second sensor, the combination of the first sensor and the first lens and the combination of the second sensor and the second lens being directed toward the same subject; and combining the first image and the second image together to form a single combined image with the second image forming a central portion of the single combined image and a peripheral portion of the first image forming a peripheral portion of the single combined image; and wherein the step of combining the first and the second images includes at least one of up-sampling the first image with a scaling factor A and down-sampling the second image with a scaling factor B; Z is the ratio of the field of view (FOV) of the first sensor to the FOV of the second sensor; and Z=A×B. ### Claim 16 (depends on 15) The method of claim 15, wherein A is a value in the range between and including 1 and Z. ### Claim 17 (depends on 15) The method of claim 15, wherein the first lens and the second lens have substantially equal f-numbers. ### Claim 18 (depends on 15) The method of claim 15, wherein the first lens and the second lens are selected such that the illuminance on the first sensor and the illuminance on the second sensor are equivalent. ### Claim 19 (depends on 15) The method of claim 15, wherein the amount of up-sampling is determined based at least in part on a digital zoom setting selected by a user. ### Claim 20 (depends on 15) The method of claim 15, wherein the central portion and the peripheral portion of the single combined image have similar levels of at least one of brightness, background noise, motion artifacts, and depth of field. ### Claim 21 (depends on 15) The method of claim 15, wherein: the second lens has been designed to work with extended depth of field (EDoF) processing by the introduction of specified aberrations into the second image formed on the second sensor; and the second image is subjected to EDoF processing to focus the second image. ### Claim 22 (depends on 21) The method of claim 21, wherein: the first image has a first depth of field determined by the combination of the first sensor and the first lens; and the EDoF processing of the second image results in the second image having a second depth of field that is substantially equal to the first depth of field. ### Claim 23 (depends on 21) The method of claim 21, wherein: the first lens has been designed to work with extended depth of field (EDoF) processing by the introduction of specified aberrations into the first image formed on the first sensor; and the first image is subjected to EDoF processing to focus the first image.