Controller and machine learning device

A machine learning device for learning how to control a conveyance operation of a conveying machine that conveys a conveyance article in relation to a state of the conveyance article during the conveyance operation,

• the machine learning device comprising:
• a state observation unit that observes, as state variables representing a current state of an environment, conveyance operation data indicating a state of the conveyance operation of the conveying machine, and conveyance article state data indicating the state of the conveyance article;
• a determination data acquisition unit that acquires, as determination data, conveyance speed determination data indicating an appropriateness determination result relating to a conveyance speed of the conveyance article, and conveyance article state determination data indicating an appropriateness determination result relating to variation in the state of the conveyance article;
• and
• a learning unit that learns the conveyance operation data and the conveyance article state data in association with each other by using the state variables and the determination data.

A learned model for causing a computer to function so as to output a value of selection of an action for controlling a conveyance operation of a conveying machine that conveys a conveyance article relative to a state of the conveyance article during the conveyance operation,

• the learned model comprising a value function for outputting, with respect to a state of an environment defined on the basis of conveyance operation data indicating a state of the conveyance operation of the conveying machine and conveyance article state data indicating the state of the conveyance article, a value of selection of an action for controlling the conveyance operation of the conveying machine that can be selected in the state of the environment, wherein
• the value function has been obtained by learning, on the basis of an appropriateness determination result relating to a conveyance speed of the conveyance article and an appropriateness determination result relating to variation in the state of the conveyance article, which are obtained by executing a predetermined control action on the conveyance operation of the conveying machine in a predetermined environment state, the selection value of the predetermined control action in the predetermined environment state;
• and
• the learned model causes the computer to function so as to execute arithmetic using the value function, with the state of the environment and the action for controlling the conveyance operation of the conveying machine as input, and output, on the basis of an arithmetic result, a value of an action for controlling the conveyance operation of the conveying machine relative to the state of the environment.