# US 9,438,888 B2 · Systems and methods for stereo imaging with camera arrays > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,888 B2 - Document ID: US9438888B2 - Status: Granted - Kind: B2 - Application number: 14/216,968 - Filed: 2014-03-17 - Published: 2014-09-18 - Granted: 2016-09-06 - Earliest claimed: 2013-03-15 - Assignees: Pelican Imaging Corporation - Applicants: Pelican Imaging Corporation - Inventors: Kartik Venkataraman; Paul Gallagher; Ankit Jain; Semyon Nisenzon - Web page: https://patentgrep.com/patent/US9438888B2 ## CPC - H04N 13/243: using three or more two-dimensional [2D] image sensors - G01P 3/38: using photographic means - G06T 7/285: using a sequence of stereo image pairs - G06T 7/55: from multiple images - G06T 7/557: from light fields, e.g. from plenoptic cameras - G06T 7/579: from motion - G06T 7/593: from stereo images - G06T 2207/10012: Stereo images - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G}) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems and methods for stereo imaging with camera arrays in accordance with embodiments of the invention are disclosed. In one embodiment, a method of generating depth information for an object using two or more array cameras that each include a plurality of imagers includes obtaining a first set of image data captured from a first set of viewpoints, identifying an object in the first set of image data, determining a first depth measurement, determining whether the first depth measurement is above a threshold, and when the depth is above the threshold: obtaining a second set of image data of the same scene from a second set of viewpoints located known distances from one viewpoint in the first set of viewpoints, identifying the object in the second set of image data, and determining a second depth measurement using the first set of image data and the second set of image data. ## Claims ### Claim 1 A method of generating depth information for an object in a scene using two or more array cameras that each include a plurality of imagers where each imager captures an image of the scene, the method comprising: obtaining a first set of image data comprising image data of a scene captured from a first set of different viewpoints; identifying an object of interest in the first set of image data; determining a first depth measurement for the object of interest using at least a portion of the first set of image data; determining whether the first depth measurement for the object of interest is above a predetermined threshold; and when the depth for the object of interest is above the predetermined threshold: obtaining a second set of image data comprising image data of the same scene from a second set of different viewpoints located known distances from at least one viewpoint in the first set of different viewpoints; identifying the object of interest in the second set of image data; and determining a second depth measurement for the object of interest using at least a portion of the first set of image data and at least a portion of the second set of image data. ### Claim 2 (depends on 1) The method of claim 1, wherein obtaining a first set of image data comprising image data of a scene captured from a first set of different viewpoints comprises capturing a first set of image data using a first plurality of imagers in a first array. ### Claim 3 (depends on 2) The method of claim 2, further comprising determining image capture settings for active imagers in the first array. ### Claim 4 (depends on 3) The method of claim 3, wherein determining image capture settings for active imagers in the first array comprises calibrating for nonlinearities in the lenses of a plurality of the first plurality of imagers. ### Claim 5 (depends on 2) The method of claim 2, wherein one of the imagers is designated as a reference camera and captures image data from a reference viewpoint. ### Claim 6 (depends on 2) The method of claim 2, wherein identifying an object of interest in the first set of image data comprises generating a preview image, presenting the preview image via a screen, and capturing user input. ### Claim 7 (depends on 2) The method of claim 2, wherein identifying an object of interest in the first set of image data comprises detecting motion in an area of the scene. ### Claim 8 (depends on 2) The method of claim 2, wherein identifying an object of interest in the first set of image data comprises detecting an object in a designated region of interest. ### Claim 9 (depends on 2) The method of claim 2, wherein identifying an object of interest in the first set of image data comprises selecting one or more pixels and associating the one or more pixels with the object. ### Claim 10 (depends on 2) The method of claim 2, wherein obtaining a second set of image data comprising image data of the same scene from a second set of different viewpoints located known distances from at least one viewpoint in the first set of different viewpoints comprises capturing a second set of image data using a second plurality of imagers in a second array. ### Claim 11 (depends on 10) The method of claim 10, wherein the second array is the first array placed in a different location from the location in which the first array captured the first set of image data. ### Claim 12 (depends on 11) The method of claim 11, further comprising estimating the baseline distance between the two arrays by cross-correlating one or more sets of corresponding pixels from the first set of image data and the second set of image data. ### Claim 13 (depends on 1) The method of claim 1, wherein determining a first depth measurement for the object of interest using at least a portion of the first set of image data comprises determining the disparity between the location of a pixel in one image in the first set of image data and a corresponding pixel in a second image in the first set of image data. ### Claim 14 (depends on 1) The method of claim 1, further comprising calculating a confidence measure for the depth of the object of interest. ### Claim 15 (depends on 1) The method of claim 1, further comprising determining if the object of interest is visible in the second set of image data and identifying the object of interest in the second set of image data when the object of interest is visible in the second set of image data. ### Claim 16 (depends on 1) The method of claim 1, wherein determining a second depth measurement for the object of interest using at least a portion of the first set of image data and at least a portion of the second set of image data comprises determining the disparity between a first pixel associated with the object of interest in at least one image in the first set of image data and a corresponding second pixel in at least one image in the second set of image data. ### Claim 17 (depends on 1) The method of claim 1, wherein determining a second depth measurement for the object of interest using at least a portion of the first set of image data and at least a portion of the second set of image data comprises calculating the disparity between the location of a pixel in one image in the first set of image data and a corresponding pixel in a second image in the second set of image data. ### Claim 18 (depends on 1) The method of claim 1, wherein determining a second depth measurement for the object of interest using at least a portion of the first set of image data and at least a portion of the second set of image data comprises utilizing the first depth measurement for the object of interest.