# US 10,500,668 B2 · Machine learning device, arc welding control device, arc welding robot system, and welding system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,500,668 B2 - Document ID: US10500668B2 - Status: Granted - Kind: B2 - Application number: 15/222,051 - Filed: 2016-07-28 - Published: 2017-02-02 - Granted: 2019-12-10 - Earliest claimed: 2015-07-31 - Assignees: Fanuc Corporation - Applicants: Fanuc Corporation - Inventors: Shigeo Yoshida; Masayoshi Mori; Kazutaka Nakayama - Web page: https://patentgrep.com/patent/US10500668B2 ## CPC - B23K 9/0956: {using sensing means, e.g. optical} - B23K 9/0953: {using computing means} - B23K 9/287: {Supporting devices for electrode holders} - B25J 9/163: {learning, adaptive, model based, rule based expert control} - Y10S 901/02: Arm motion controller - Y10S 901/42: Welding - B23K: SOLDERING OR UNSOLDERING; WELDING; CLADDING OR PLATING BY SOLDERING OR WELDING; CUTTING BY APPLYING HEAT LOCALLY, e.g. FLAME CUTTING; WORKING BY LASER BEAM - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A machine learning device which learns to determine at least one arc welding condition includes a state observation unit which observes a state variable consisting of at least one physical quantity regarding the arc welding and the at least one arc welding condition at least during or after the arc welding, and a learning unit which learns a change in the at least one physical quantity observed by the state observation unit and the at least one arc welding condition in association with each other. ## Claims ### Claim 1 An arc welding control device, comprising: a machine learning device which learns to determine at least one arc welding condition; and an arc welding execution unit which executes arc welding in accordance with the determined at least one arc welding condition output from the machine learning device, the machine learning device comprising: a state observation unit which observes a state variable comprising (i) at least one physical quantity regarding the arc welding at least during or after the arc welding and (ii) the at least one arc welding condition, wherein the at least one arc welding condition includes one of an amount of adjustment of a weld waveform, push/drag angles of a welding torch, an aim angle of a welding torch, a torch aim position, a weaving condition, an arc sensor condition, and an offset amount of the welding position in a multi-layer welding, and wherein the at least one physical quantity includes at least one of an outer appearance of a weld bead, an excess weld metal height of the weld bead, a bead width of the weld bead, which are obtained by processing image data picked up by an image pickup unit, an amount of generated spatter, and an amount of penetration obtained by a penetration measurement unit; and a learning unit which learns to determine the at least one arc welding condition by updating a function that determines the at least one arc welding condition based on the state variable, wherein the learning unit comprises (i) a reward computation unit which computes a reward for a result of the determined at least one arc welding condition and (ii) a function update unit which updates the function based on the reward computed by the reward computation unit, wherein the at least one arc welding condition is learned so as to increase the reward computed by the reward computation unit, by repeatedly carrying out the updating of the function in the function update unit. ### Claim 2 (depends on 1) The arc welding control device according to claim 1, wherein the reward computation unit includes a reward condition setting unit which sets a reward condition and computes the reward based on the reward condition set by the reward condition setting unit. ### Claim 3 (depends on 1) The arc welding control device according to claim 1, wherein the learning unit further comprises a learning result storage unit which stores a learning result that the function update unit has learned. ### Claim 4 (depends on 1) The arc welding control device according to claim 1, further comprising: a decision making unit which decides the at least one arc welding condition and an amount of adjustment of the at least one arc welding condition, from the current state variable, based on a learning result of the learning unit. ### Claim 5 (depends on 4) The arc welding control device according to claim 4, wherein the learning unit learns or repeatedly learns the adjustment of the at least one arc welding condition during the execution of arc welding. ### Claim 6 (depends on 4) An arc welding robot system, comprising: a robot which carries out arc welding, a physical quantity measurement unit which measures at least one physical quantity regarding the arc welding at least during or after the execution of the arc welding by the robot, and an arc welding control device according to claim 4. ### Claim 7 (depends on 6) The arc welding robot system according to claim 6, further comprising a positioner which changes the posture of a workpiece to be welded, wherein the robot and the positioner carry out arc welding while performing operations associated with each other. ### Claim 8 (depends on 6) A welding system, comprising: a plurality of arc welding robot systems according to claim 6, and a communication unit which mutually connects the plurality of arc welding robot systems, wherein the communication unit transmits and receives at least one of at least one physical quantity observed by at least one state observation unit and the learning result stored in at least one learning unit among the plurality of arc welding robot systems. ### Claim 9 (depends on 8) The welding system according to claim 8, further comprising a host computer connected to the communication unit, wherein at least one of the at least one physical quantity observed by the at least one state observation unit and the learning result stored in the at least one learning unit among the plurality of arc welding robot systems is stored in the host computer. ### Claim 10 (depends on 1) The arc welding control device according to claim 1, wherein the function update unit updates the function by using the following formula Q ⁡ ( s t , a t ) ← Q ⁡ ( s t , a t ) + α ⁡ ( r t + 1 + γmax a ⁢ Q ⁡ ( s t + 1 , a ) - Q ⁡ ( s t , a t ) ) ( 1 ) where st and at represent environment and action at time t, respectively, Q(s,a) is a value of selecting action a under environmental s, r is the reward computed in accordance with a change of the environment, γ represents a discount factor which satisfies 0<γ≤1, and α represents a learning factor which satisfies 0<α≤1.