# US 10,484,666 B1 · Method and apparatus for a computer vision camera unit > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,666 B1 - Document ID: US10484666B1 - Status: Granted - Kind: B1 - Application number: 15/462,707 - Filed: 2017-03-17 - Granted: 2019-11-19 - Earliest claimed: 2016-03-17 - Assignees: Herman Herman - Applicants: Edge Robotics Inc. - Inventors: Herman Herman - Web page: https://patentgrep.com/patent/US10484666B1 ## CPC - H04N 13/25: using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - H04N 5/2226: {Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10)} - H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) - H04N 13/246: Calibration of cameras - H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps - H10F 39/8063: {Microlenses} - G06T 2207/10012: Stereo images - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H10F: INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION ## Abstract A computer vision camera unit comprising first and second pairs, of image capture assemblies, both of which have an approximately same orientation to a first axis. The first and second pairs are optically focused to capture first and second fields of view that, at least, overlap each other. Providing the first and second pairs with imager chips of differing light sensitivity permits high dynamic range to be realized. Having the second field of view within the first field of view permits specialization, between the first and second pairs, similar to the roles of peripheral and foveal vision, found in biological stereo vision systems. The image capture assemblies can be organized in a rectangular configuration that implies third and fourth pairs, orthogonal to the first and second pairs. The orthogonal but interlocking nature, of the first and second pairs in relation to the third and fourth pairs, enables an autocalibration technique. ## Claims ### Claim 1 A method for a first computer vision camera unit, comprising: identifying a same first feature with first, second, third, and fourth image capture assemblies, each image capture assembly having a lens assembly embedded, in a first frame, at a vertex of a first rectangular configuration; identifying, with first, second, third, and fourth imager chips, of, respectively, the first, second, third, and fourth image capture assemblies, the same first feature as being at first, second, third, and fourth locations, relative to, respectively, first, second, third, and fourth principal points; determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to a first axis, there is a first misalignment between the first and second locations; determining, performed at least in part with a configuration of computing hardware and programmable memory, if there is the first misalignment, a first change, for each of the first and second principal points, that reduces the misalignment to zero; determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to the first axis, there is second misalignment between the third and fourth locations; determining, performed at least in part with a configuration of computing hardware and programmable memory, if there is a second misalignment, a second change, for each of the third and fourth principal points, that reduces the misalignment to zero; determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to a second axis, there is third misalignment between the first and third locations; determining, performed at least in part with a configuration of computing hardware and programmable memory, if there is a third misalignment, a third change, for each of the first and third principal points, that reduces the misalignment to zero; determining, performed at least in part with a configuration of computing hardware and programmable memory, if, with respect to the second axis, there is a fourth misalignment between the second and fourth locations; determining, performed at least in part with a configuration of computing hardware and programmable memory, if there is a fourth misalignment, a fourth change, for each of the second and fourth principal points, that reduces the misalignment to zero; determining, performed at least in part with a configuration of computing hardware and programmable memory, as a result of the first and third changes, a first distance by which the first principal point changes its location; determining, performed at least in part with a configuration of computing hardware and programmable memory, as a result of the first and fourth changes, a second distance by which the second principal point changes its location; determining, performed at least in part with a configuration of computing hardware and programmable memory, as a result of the second and third changes, a third distance by which the third principal point changes its location; determining, performed at least in part with a configuration of computing hardware and programmable memory, as a result of the second and fourth changes, a fourth distance by which the fourth principal point changes its location; determining, performed at least in part with a configuration of computing hardware and programmable memory, the first, second, third, and fourth changes such that a sum, of first, second, third, and fourth distances, is minimized; determining, performed at least in part with a configuration of computing hardware and programmable memory, a first depth, of the first feature from the first computer vision camera unit, based upon a change of location of the first feature, along the second axis, between the first and second locations; determining, performed at least in part with a configuration of computing hardware and programmable memory, a second depth, of the first feature from the first computer vision camera unit, based upon a change of location of the first feature, along the first axis, between the first and third locations; and rejecting, performed at least in part with a configuration of computing hardware and programmable memory, a set, of first, second, third, and fourth changes, if the first depth does not equal the second depth. ### Claim 2 A method for a first computer vision camera unit, comprising: capturing a first pair of images, for use with stereo depth perception, from a first pair of image capture assemblies oriented according to a first axis and optically focused to capture a first field of view; capturing a second pair of images, for use with stereo depth perception, from a second pair of image capture assemblies oriented according to the first axis and optically focused to capture a second field of view; wherein the first and second fields of view are optically focused to overlap; wherein the first and second fields of view are optically focused such that the second field of view is within the first field of view; capturing the first pair of images with a first pair of imager chips with a first density of photo-sensitive elements; capturing the second pair of images with a second pair of imager chips with a second density of photo-sensitive elements; and wherein the first and second densities are different and are selected such that a second resolution, of the second pair of images, is greater than a first resolution of the first pair of images. ### Claim 3 A method for a first computer vision camera unit comprising: capturing a first pair of images, for use with stereo depth perception, from a first pair of image capture assemblies oriented according to a first axis and optically focused to capture a first field of view; capturing a second pair of images, for use with stereo depth perception, from a second pair of image capture assemblies oriented according to the first axis and optically focused to capture a second field of view; wherein the first and second fields of view are optically focused to overlap; wherein the first and second fields of view are optically focused such that the second field of view is smaller than the first field of view; capturing the first pair of images with a first pair of imager chips; capturing the second pair of images with a second pair of imager chips; capturing at a first rate, with the first pair of imager chips, a first series of pairs of images; capturing at a second rate, with the second pair of imager chips, a second series of pairs of images; and wherein the first rate is higher than the second rate. ### Claim 4 (depends on 3) The method of claim 3, wherein the first rate is selected to better capture motion while the second rate is chosen to better capture detail. ### Claim 5 (depends on 3) The method of claim 3, wherein the first series, of pairs of images, is captured as monochrome, while the second series, of pairs of images, includes a capture of color. ### Claim 6 A method for a first computer vision camera unit, comprising: capturing a first pair of images, for use with stereo depth perception, from a first pair of image capture assemblies oriented according to a first axis and optically focused to capture a first field of view; capturing a second pair of images, for use with stereo depth perception, from a second pair of image capture assemblies oriented according to the first axis and optically focused to capture a second field of view; wherein the first and second fields of view are optically focused to overlap; capturing the first pair of images with a first pair of imager chips with a first level of light sensitivity for its photo-sensitive elements; capturing the second pair of images with a second pair of imager chips with a second level of light sensitivity for its photo-sensitive elements; and wherein the first and second light sensitivity levels are selected to be substantially different from each other. ### Claim 7 (depends on 6) The method of claim 6, wherein the first and second light sensitivity levels are selected to provide the first computer vision camera unit with a high dynamic range. ### Claim 8 (depends on 1) The method of claim 1, wherein the first, second, third, and fourth changes are determined, for minimization of their sum, by a non-heuristic iterative method. ### Claim 9 (depends on 1) The method of claim 1, wherein the first, second, third, and fourth changes are determined, for minimization of their sum, by an at least partially stochastic method. ### Claim 10 (depends on 2) The method of claim 2, wherein the first and second pairs of image capture assemblies form a rectangular configuration. ### Claim 11 (depends on 3) The method of claim 3, wherein the first and second pairs of image capture assemblies form a rectangular configuration. ### Claim 12 (depends on 6) The method of claim 6, wherein the first and second pairs of image capture assemblies form a rectangular configuration.