# US 2019/0314996 A1 · Control device and machine learning device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0314996 A1
- Document ID: US20190314996A1
- Status: Published application
- Kind: A1
- Application number: 16/267,841
- Filed: 2019-02-05
- Published: 2019-10-17
- Earliest claimed: 2018-02-09
- Assignees: Fanuc Corporation
- Applicants: Fanuc Corporation
- Inventors: Hideki Otsuki; Hiroshi Abe
- Web page: https://patentgrep.com/patent/US20190314996A1

## CPC

- B25J 11/008: {Manipulators for service tasks}
- B25J 9/1661: {characterised by task planning, object-oriented languages}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/092: Reinforcement learning
- G06N 20/00: Machine learning
- G05B 2219/39295: Learn position correction values to be added to reference values
- G05B 2219/40053: Pick 3-D object from pile of objects
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A machine learning device includes: a state observation unit that observes component arrangement data representing an arrangement of components on a component serving place, component data representing information of the components, and operator status data representing status information of an operator, as state variables representing a current state of an environment; a determination data acquisition unit that acquires product quality determination data for determining quality of the product which is assembled based on an arrangement of the components and takt time determination data for determining takt time for assembly of the product as determination data; and a learning unit that performs learning based on the state variables and the determination data in a manner to associate information of the components used for assembling the product and status information of the operator with respect to an arrangement of the components on the component serving place.

## Claims

### Claim 1

A control device that instructs an arrangement of components on a component serving place, the arrangement being performed by a serving robot, the control device comprising:
a machine learning device that learns a component arrangement on the component serving place by the serving robot, wherein
the machine learning device includes
a state observation unit that observes component arrangement data representing an arrangement of the components on the component serving place, component data representing information of the components, and operator status data representing status information of an operator who assembles a product with the components, as state variables representing a current state of an environment,
a determination data acquisition unit that acquires product quality determination data for determining quality of the product which is assembled based on an arrangement of the components and takt time determination data for determining takt time for assembly of the product as determination data representing a suitability determination result on assembly of the product by the operator, and
a learning unit that performs learning based on the state variables and the determination data in a manner to associate information of the components used for assembling the product and status information of the operator with respect to an arrangement of the components on the component serving place.

### Claim 2 (depends on 1)

The control device according to claim 1, wherein
the learning unit includes
a reward calculation unit that obtains reward related to the suitability determination result, and
a value function update unit that updates a function representing a value of an arrangement of the components on the component serving place with respect to information of the components used for assembling the product and status information of the operator, by using the reward, and
the reward calculation unit imparts higher reward as quality of the product is higher and the takt time is shorter.

### Claim 3 (depends on 1)

The control device according to claim 1, wherein the learning unit calculates the state variables and the determination data in a multilayer structure.

### Claim 4

A control device that instructs an arrangement of components on a component serving place, the arrangement being performed by a serving robot, the control device comprising:
a machine learning device that has learned a component arrangement on the component serving place by the serving robot, wherein
the machine learning device includes
a state observation unit that observes component arrangement data representing an arrangement of the components on the component serving place, component data representing information of the components, and operator status data representing status information of an operator who assembles a product with the components, as state variables representing a current state of an environment,
a learning unit that has performed learning in a manner to associate information of the components used for assembling the product and status information of the operator with respect to an arrangement of the components on the component serving place, and
a decision-making unit that determines an arrangement of the components on the component serving place based on state variables observed by the state observation unit and a learning result obtained by the learning unit.

### Claim 5 (depends on 1)

The control device according to claim 1, wherein the machine learning device exists in a cloud server.

### Claim 6

A machine learning device that learns an arrangement of components on a component serving place, the arrangement being performed by a serving robot, the machine learning device comprising:
a state observation unit that observes component arrangement data representing an arrangement of the components on the component serving place, component data representing information of the components, and operator status data representing status information of an operator who assembles a product with the components, as state variables representing a current state of an environment;
a determination data acquisition unit that acquires product quality determination data for determining quality of the product which is assembled based on an arrangement of the components and takt time determination data for determining takt time for assembly of the product as determination data representing a suitability determination result on assembly of the product by the operator; and
a learning unit that performs learning based on the state variables and the determination data in a manner to associate information of the components used for assembling the product and status information of the operator with respect to an arrangement of the components on the component serving place.

### Claim 7

A machine learning device that has learned an arrangement of components on a component serving place, the arrangement being performed by a serving robot, the machine learning device comprising:
a state observation unit that observes component arrangement data representing an arrangement of the components on the component serving place, component data representing information of the components, and operator status data representing status information of an operator who assembles a product with the components, as state variables representing a current state of an environment;
a learning unit that has performed learning in a manner to associate information of the components used for assembling the product and status information of the operator with respect to an arrangement of the components on the component serving place; and
a decision-making unit that determines an arrangement of the components on the component serving place based on state variables observed by the state observation unit and a learning result obtained by the learning unit.
