# US 11,113,844 B2 · Systems and methods for automatic quality assessment of multiple-camera calibration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,844 B2 - Document ID: US11113844B2 - Status: Granted - Kind: B2 - Application number: 16/695,791 - Filed: 2019-11-26 - Published: 2021-05-27 - Granted: 2021-09-07 - Earliest claimed: 2019-11-26 - Assignees: The Boeing Company - Applicants: The Boeing Company - Inventors: Sildomar T. Monteiro; Willaredt E. Hardy; Anthony W. Baker - Web page: https://patentgrep.com/patent/US11113844B2 ## CPC - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 7/85: {Stereo camera calibration} - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]} - G06V 10/761: {Proximity, similarity or dissimilarity measures} - H04N 13/246: Calibration of cameras - G06T 2200/08: involving all processing steps from image acquisition to 3D model generation - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes - H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) - H04N 13/243: using three or more two-dimensional [2D] image sensors - H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems and methods for assessing the calibration of an array of cameras. The method including inputting into a processor captured images from at least two cameras of the array of cameras, the captured images having features from an image. The method further including extracting one or more extracted features from the captured images, matching one or more extracted features between pairs of the at least two cameras to create a set of matched features, selecting matching points from the set of matched features, generating a three-dimensional reconstruction of objects in a field of view of the at least two cameras, and outputting the three-dimensional reconstruction wherein the three-dimensional reconstruction comprises indicators of calibration errors. ## Claims ### Claim 1 A method for assessing calibration of an array of cameras, the method comprising: inputting captured images from at least two cameras of the array of cameras, the captured images comprising features from an image; extracting one or more extracted features from the captured images; matching one or more extracted features between pairs of the at least two cameras to create a set of matched features; selecting matching points from the set of matched features by: selecting matching points based on a distance ratio; and applying a homography transform with RANSAC (RANdom SAmple Consensus); generating a three-dimensional reconstruction of objects in a field of view of the at least two cameras; and outputting the three-dimensional reconstruction wherein the three-dimensional reconstruction comprises indicators of calibration errors. ### Claim 2 (depends on 1) The method of claim 1 wherein the step of extracting one or more extracted features further comprises extracting using at least one of: ORB (Oriented FAST (Features from Accelerated Segment Test) and Rotated BRIEF (Binary Robust Independent Elementary Features)) feature detection; and SURF (Speeded Up Robust Features) feature detection. ### Claim 3 (depends on 1) The method of claim 1 wherein the step of extracting one or more extracted features further comprises: selecting a subset of candidate features to extract; and extracting using ANMS (Adaptive Non-Maximal Suppression) feature detection. ### Claim 4 (depends on 3) The method of claim 3 wherein the step of extracting using ANMS feature detection further comprises extracting using SSC (Suppression via Square Covering) feature detection. ### Claim 5 (depends on 1) The method of claim 1 wherein the step of matching one or more extracted features further comprises at least one of: matching through brute force matching; and matching through Fast Library for Approximate Nearest Neighbors (FLANN) approximate matching. ### Claim 6 (depends on 1) The method of claim 1 wherein the step of generating a three-dimensional reconstruction further comprises: producing a three-dimensional point cloud; and producing a reconstruction error rate. ### Claim 7 (depends on 6) The method of claim 6 wherein the step of outputting the three-dimensional reconstruction further comprises at least one of: outputting the three-dimensional point cloud; outputting the reconstruction error rate; and outputting a visualization of the matched features and wherein the indicators of calibration errors further comprise lines. ### Claim 8 (depends on 1) The method of claim 1 further comprising: projecting the image onto the objects in the field of view of the at least two cameras. ### Claim 9 A system for assessing calibration of an array of cameras, the system comprising: at least one processor; and a memory storing a set of instructions that when executed by the processor cause the processor to initiate operations comprising: inputting captured images from at least two cameras of the array of cameras, the captured images comprising features from an image; extracting one or more extracted features from the captured images; matching one or more extracted features between pairs of the at least two cameras to create a set of matched features; selecting matching points from the set of matched features by: selecting matching points based on a distance ratio; and applying a homography transform with RANSAC (RANdom SAmple Consensus); generating a three-dimensional reconstruction of objects in a field of view of the at least two cameras; and outputting the three-dimensional reconstruction wherein the three-dimensional reconstruction comprises indicators of calibration errors. ### Claim 10 (depends on 9) The system of claim 9 wherein the operation of extracting one or more extracted features further comprises at least one of: extracting using ORB (Oriented FAST (Features from Accelerated Segment Test) and Rotated BRIEF (Binary Robust Independent Elementary Features)) feature detection; and extracting using SURF (Speeded Up Robust Features) feature detection. ### Claim 11 (depends on 9) The system of claim 9 wherein the operation of extracting one or more extracted features further comprises: selecting a subset of candidate features to extract; and extracting using ANMS (Adaptive Non-Maximal Suppression) feature detection. ### Claim 12 (depends on 11) The system of claim 11 wherein the operation of extracting using ANMS feature detection further comprises extracting using SSC (Suppression via Square Covering) feature detection. ### Claim 13 (depends on 9) The system of claim 9 wherein the operation of matching one or more extracted features further comprises at least one of: matching through brute force matching; and matching through Fast Library for Approximate Nearest Neighbors (FLANN) approximate matching. ### Claim 14 (depends on 9) The system of claim 9 wherein the operation of generating a three-dimensional reconstruction further comprises: producing a three-dimensional point cloud; and producing a reconstruction error rate. ### Claim 15 (depends on 14) The system of claim 14 wherein the operation of outputting the three-dimensional reconstruction further comprises at least one of: outputting the three-dimensional point cloud; outputting the reconstruction error rate; and outputting a visualization of the matched features and wherein the indicators of calibration errors further comprise lines. ### Claim 16 (depends on 9) The system of claim 9 further comprising: a projector for projecting the image onto the objects in the field of view of the at least two cameras. ### Claim 17 (depends on 9) The system of claim 9 wherein the operation of extracting one of more extracted features further comprises executing a feature extraction function with one or more variable parameters. ### Claim 18 (depends on 17) The system of claim 17 wherein the one or more variable parameters comprise at least one of: a maximum number of features to be detected; an ORB (Oriented FAST (Features from Accelerated Segment Test) and Rotated BRIEF (Binary Robust Independent Elementary Features)) detector pyramid decimation ratio; an ORB detector number of pyramid levels; an ORB detector FAST threshold; a percentage of maximum number of features to retain in an ANMS (Adaptive Non-Maximal Suppression) algorithm; a maximum distance between the best and second-best match; and a maximum reprojection error to reject points as outliers. ### Claim 19 (depends on 7) The method of claim 7 wherein the step of outputting a visualization of the matched features comprises using circles to indicate matched feature locations. ### Claim 20 (depends on 15) The system of claim 15 wherein outputting a visualization of the matched features comprises using circles to indicate matched feature locations.