# US 9,519,736 B2 · Data generation device for vision sensor and detection simulation system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,736 B2 - Document ID: US9519736B2 - Status: Granted - Kind: B2 - Application number: 14/602,816 - Filed: 2015-01-22 - Published: 2015-07-23 - Granted: 2016-12-13 - Earliest claimed: 2014-01-23 - Assignees: Fanuc Corporation - Applicants: FANUC Corporation - Inventors: Hiroyuki Atohira; Yoshiharu Nagatsuka - Web page: https://patentgrep.com/patent/US9519736B2 ## CPC - G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) - B25J 9/1671: {characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems} - G05B 2219/39031: Use of model for robot and for measuring device - Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS] - Y10S 901/47: Optical - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A data generation device and a detection simulation system, capable of executing offline simulation using a vision sensor, without calibration for associating the vision sensor with the robot. The device has: a placing part which places three-dimensional models of a robot, a vision sensor and an object in a virtual space; a robot placement position designating part and a vision sensor placement position designating part which designate placement positions of the robot and the vision sensor in the virtual space, respectively, based on a reference coordinate system defined in the virtual space; a placement position storing part which stores the placement positions; a positional relationship calculating part which calculates a positional relationship between a robot coordinate system and a sensor coordinate system; and a positional relationship storing part which stores the positional relationship as data used in a detection process of the vision sensor. ## Claims ### Claim 1 A data generation device configured to generate data used in a detection process using a vision sensor, based on positions of a three-dimensional model of a robot, a three-dimensional model of the vision sensor and a three-dimensional model of an object which are positioned in a virtual space, the data generation device comprising: a robot placement position designating part configured to designate a placement position of the three-dimensional model of the robot in the virtual space based on a reference coordinate system defined in the virtual space; a vision sensor placement position designating part configured to designate a placement position of the three-dimensional model of the vision sensor in the virtual space based on the reference coordinate system; a placement position storing part configured to store the placement positions of the three-dimensional model of the robot and the three-dimensional model of the vision sensor; a positional relationship calculating part configured to calculate a positional relationship between a robot coordinate system defined in relation to the three-dimensional model of the robot and a sensor coordinate system defined in relation to the three-dimensional model of the vision sensor, based on the placement positions of the three-dimensional model of the robot and the three-dimensional model of the vision sensor, without carrying out a calibration for associating the robot with the vision sensor; and a positional relationship storing part configured to store the positional relationship between the robot coordinate system and the sensor coordinate system as data used in a detection process of the vision sensor. ### Claim 2 (depends on 1) The data generation device as set forth in claim 1, further comprising: an object position designating part configured to designate a position of the three-dimensional model of the object in the virtual space based on the reference coordinate system; and a CG image generating part configured to generate a computer graphics image which is a quasi-image corresponding to an image obtained when the object is captured by the vision sensor, based on a positional relationship between the vision sensor and the object. ### Claim 3 (depends on 2) A detection simulation system, comprising: the data generation device as set forth in claim 2; and an image processor configured to carry out a detection process of the object by using the data used in the detection process of the vision sensor. ### Claim 4 (depends on 3) The detection simulation system as set forth in claim 3, the image processor comprising: a detection processing part configured to carry out the detection process the object in relation to the computer graphics image by using the data; and an object position calculating part configured to calculate a position of the object based on a result of the detection process of the object carried out by the detection processing part. ### Claim 5 (depends on 3) The detection simulation system as set forth in claim 3, wherein the three-dimensional model of the vision sensor is attached to a movable part of the three-dimensional model of the robot. ### Claim 6 (depends on 1) The data generation device as set forth in claim 1, wherein the three-dimensional model of the vision sensor is attached to a movable part of the three-dimensional model of the robot.