# US 2017/0285584 A1 · Machine learning device that performs learning using simulation result, machine system, manufacturing system, and machine learning method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0285584 A1 - Document ID: US20170285584A1 - Status: Published application - Kind: A1 - Application number: 15/472,395 - Filed: 2017-03-29 - Published: 2017-10-05 - Earliest claimed: 2016-04-04 - Assignees: Fanuc Corporation - Applicants: Fanuc Corporation - Inventors: Hiroshi NAKAGAWA; Takafumi KAJIYAMA - Web page: https://patentgrep.com/patent/US20170285584A1 ## CPC - G05B 13/027: {using neural networks only} - B25J 9/1664: {characterised by motion, path, trajectory planning} - B25J 9/1671: {characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems} - G05B 2219/39298: Trajectory learning - G05B 2219/40515: Integration of simulation and planning - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) ## Abstract A machine learning device that learns a control command for a machine by machine learning, including a machine learning unit that performs the machine learning to output the control command; a simulator that performs a simulation of a work operation of the machine based on the control command; and a first determination unit that determines the control command based on an execution result of the simulation by the simulator. ## Claims ### Claim 1 A machine learning device that learns a control command for a machine by machine learning, comprising: a machine learning unit that performs the machine learning to output the control command; a simulator that performs a simulation of a work operation of the machine based on the control command; and a first determination unit that determines the control command based on an execution result of the simulation by the simulator. ### Claim 2 (depends on 1) The machine learning device according to claim 1, wherein the first determination unit determines that the control command is good and performs inputting of the control command to the machine when there is no problem in the execution result of the simulation, and determines that the control command is bad and stops the inputting of the control command to the machine when there is a problem in the execution result of the simulation. ### Claim 3 (depends on 2) The machine learning device according to claim 2, wherein when a determination result by the first determination unit is bad, the inputting of the control command to the machine is stopped, and learning is performed by providing a first result label obtained from the first determination unit to the machine learning unit as training data. ### Claim 4 (depends on 3) The machine learning device according to claim 3, wherein when the determination result by the first determination unit is bad, the learning is performed by providing, along with the first result label, a first state quantity including part or whole of calculation result data obtained from the simulator to the machine learning unit. ### Claim 5 (depends on 2) The machine learning device according to claim 2, further comprising a second determination unit that determines a work result of the machine by the control command, wherein when a determination result by the first determination unit is good, inputting of the control command to the machine is performed, and learning is performed by providing a second result label obtained from the second determination unit to the machine learning unit as training data. ### Claim 6 (depends on 5) The machine learning device according to claim 5, wherein when the determination result by the first determination unit is good, the learning is performed by providing to the machine learning unit, along with the second result label, a second state quantity including at least one of output data of a sensor that detects a state of the machine or a surrounding environment, internal data of a control software that controls the machine, and calculation data obtained based on the output data of the sensor or the internal data of the control software. ### Claim 7 (depends on 2) The machine learning device according to claim 2, wherein: a state of the machine learning unit is preserved as a first state regularly or when a pre-specified condition is satisfied; and the state of the machine learning unit is returned to the first state when a frequency that the determination result by the first determination unit becomes good decreases. ### Claim 8 (depends on 2) The machine learning device according to claim 2, wherein when the determination result by the first determination unit takes three or more states including good and bad, a command speed of the machine included in the control command is changed based on the state. ### Claim 9 (depends on 2) The machine learning device according to claim 2, wherein the machine learning unit comprises: a reward calculation unit that calculates a reward based on a work state of the machine; and a value function updating unit having a value function that determines a value for the control command and updating the value function based on the reward. ### Claim 10 (depends on 1) The machine learning device according to claim 1, wherein the machine learning device comprises a neural network. ### Claim 11 (depends on 1) A machine system comprising the machine learning device according to claim 1 and the machine, the machine system comprising: a control apparatus that controls the machine; and a computer apparatus having the machine learning device including the machine learning unit, the simulator, and the first determination unit, wherein the computer apparatus and one or more the machines are connected via a network. ### Claim 12 (depends on 11) The machine system according to claim 11, wherein the learning of the machine learning unit is performed continuously even during an actual production work by the machine, or performed in advance but not performed during the actual production work by the machine. ### Claim 13 (depends on 11) A manufacturing system comprising a plurality of the machine systems according to claim 11, wherein the machine learning device is provided in each of the machine systems; and a plurality of the machine learning devices provided in the plurality of the machine systems are configured to mutually share or exchange data via a communication medium. ### Claim 14 (depends on 13) The manufacturing system according to claim 13, wherein the machine learning device exists on a cloud server. ### Claim 15 A machine learning method that learns a control command for a machine by machine learning, the machine learning method comprising: performing the machine learning to output the control command; executing a simulation of a work operation of the machine based on the control command; determining the control command based on a result of the execution of the simulation; determining that the control command is good and performing inputting of the control command to the machine when there is no problem in the execution result of the simulation; and determining that the control command is bad and stopping the inputting of the control command to the machine when there is a problem in the execution result of the simulation.