# US 11,077,557 B2 · System and method for robust calibration between a machine vision system and a robot > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,077,557 B2 - Document ID: US11077557B2 - Status: Granted - Kind: B2 - Application number: 16/192,233 - Filed: 2018-11-15 - Published: 2019-03-21 - Granted: 2021-08-03 - Earliest claimed: 2010-05-14 - Assignees: Cognex Corporation - Applicants: Cognex Corporation - Inventors: Aaron S. Wallack; Lifeng Liu; Xiangyun Ye - Web page: https://patentgrep.com/patent/US11077557B2 ## CPC - B25J 9/1692: {Calibration of manipulator} - B25J 9/1607: {Calculation of inertia, jacobian matrixes and inverses} - B25J 9/1697: {Vision controlled systems} - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G05B 2219/39008: Fixed camera detects reference pattern held by end effector - G05B 2219/39022: Transform between measuring and manipulator coordinate system - G05B 2219/39045: Camera on end effector detects reference pattern - G05B 2219/39057: Hand eye calibration, eye, camera on hand, end effector - Y10S 901/09: Closed loop, sensor feedback controls arm movement - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A system and method for robustly calibrating a vision system and a robot is provided. The system and method enables a plurality of cameras to be calibrated into a robot base coordinate system to enable a machine vision/robot control system to accurately identify the location of objects of interest within robot base coordinates. ## Claims ### Claim 1 A method for calibration between a machine vision system and a robot, the method comprising: obtaining a set of pairs of camera poses and robot poses; analyzing corresponding robot motions of the robot and camera motions of a camera among the obtained set of pairs; detecting outliers based on the analyzing; re-ordering the set of pairs, with the detected outliers removed, to obtain a set of pairs of camera poses and robot poses with suitable motions; utilizing the set of pairs of camera poses and robot poses with suitable motions to perform calibration by obtaining a plurality of images of a calibration object in at least two pairs of camera poses and robot poses and minimizing discrepancies between features of the calibration object identified in the plurality of images and features of the calibration object predicted by the robot poses in the set of pairs of camera poses and robot poses using a camera coordinate system for the camera, a calibration object coordinate system for the calibration object, a robot coordinate system for a portion of the robot that is stationary, and an effector coordinate system for a portion of the robot that moves to occupy the robot poses. ### Claim 2 (depends on 1) The method of claim 1 wherein suitable motions comprises rotation of at least five degrees. ### Claim 3 (depends on 1) The method of claim 1 wherein suitable motions comprises rotation along at least two non-parallel axes. ### Claim 4 (depends on 1) The method of claim 1 wherein the set of pairs comprises at least three pairs of camera poses and robot poses. ### Claim 5 (depends on 1) The method of claim 1 wherein the calibration object comprises a calibration plate. ### Claim 6 (depends on 1) The method of claim 1 wherein the suitable motions comprises changes between a particular pair of a reduced set of obtained images. ### Claim 7 (depends on 1) The method of claim 1 wherein the camera is affixed to an articulated arm of the robot. ### Claim 8 (depends on 1) The method of claim 1 wherein the camera is fixed in space and in view of the robot. ### Claim 9 (depends on 1) The method of claim 1 wherein the calibration object is affixed to an end effector of the robot. ### Claim 10 (depends on 1) The method of claim 1 wherein the calibration object is fixed in space. ### Claim 11 (depends on 1) The method of claim 1 wherein each outlier in the obtained set of pairs is detected based on a threshold discrepancy between the robot motion and camera motion corresponding to the outlier. ### Claim 12 (depends on 1) The method of claim 1 wherein the machine vision system comprises multiple cameras for obtaining the plurality of images. ### Claim 13 (depends on 1) The method of claim 1 wherein the calibration object comprises a first object feature and a second object feature, the first object feature and the second object feature located at a fixed, known distance relative to one another.