Apparatus and method of using target setting data for learning of an operation model

An apparatus comprising:

• a first supply unit that uses the at least one processor to supply, in response to an operation plan of a piece of equipment being newly input, the operation plan to a target setting model configured to output a corresponding target range among target setting data that is to be used for learning of an operation model configured to operate the piece of equipment, the target setting data including identification information of a parameter for which a target range is to be set among parameters relating to the piece of equipment and the target range that is set for a parameter wherein the target setting model uses machine learning to generate the target range of the target setting data;
• a second acquisition unit that uses the at least one processor to acquire output data from the target setting model supplied with the operation plan by the first supply unit;
• and
• a second learning processing unit that uses the at least one processor to perform learning processing of the operation model configured to operate the piece of equipment by using the target setting data corresponding to the output data acquired by the second acquisition unit;
• wherein the second learning processing unit uses the at least one processor to perform the learning processing of the operation model by reinforcement learning, the learning processing of the operation model including applying a preset reward function that varies a reward value according to a degree that a value of a state parameter relating to the piece of equipment operated using a control parameter output from the operation model deviates from the target range of the target setting data.

A method comprising:

• firstly acquiring an operation plan of a piece of equipment, and at least identification information of a parameter among target setting data used for learning of an operation model configured to operate the piece of equipment, the target setting data including identification information of a parameter for which a target range is to be set among parameters relating to the piece of equipment and a target range that is set for a parameter;
• firstly performing, by using learning data including the identification information and the operation plan acquired by the firstly acquiring, learning processing of a target setting model configured to output the target range of the target setting data that is used for learning of the operation model, in response to the operation plan being input wherein the learning processing of the target setting model is performed using machine learning;
• and
• secondly performing learning processing to perform, by using learning data including a value of a state parameter relating to the piece of equipment and a value of a control parameter of the piece of equipment and the target setting data, learning processing of the operation model so as to output a value of a control parameter from the operation model causing a state relating to the piece of equipment to be closer to a state corresponding to a content of the target setting data, in response to a value of the state parameter being input;
• wherein the learning processing of the operation model is performed by reinforcement learning, the learning processing of the operation model including applying a preset reward function that varies a reward value according to a degree that a value of the state parameter relating to the piece of equipment operated using the value of the control parameter outputted from the operation model deviates from the target range of the target setting data.

A method comprising:

• firstly supplying, in response to an operation plan of a piece of equipment being newly input, the operation plan to a target setting model configured to output a target range of a parameter among target setting data that is to be used for learning of an operation model configured to operate the piece of equipment, the target setting data including identification information of a parameter for which a target range is to be set among parameters relating to the piece of equipment and the target range that is set for a parameter wherein the target setting model uses machine learning to generate the target range of the target setting data;
• secondly acquiring output data from the target setting model supplied with the operation plan by the firstly supplying;
• and
• secondly performing learning processing of the operation model configured to operate the piece of equipment by using learning data including the target setting data corresponding to the output data acquired by the secondly acquiring;
• wherein the learning processing of the operation model is performed by reinforcement learning, the learning processing of the operation model including applying a preset reward function that varies a reward value according to a degree that a value of a state parameter relating to the piece of equipment operated using a control parameter output from the operation model deviates from the target range of the target setting data.

A non-transitory computer readable medium having recorded thereon one or more programs that cause a computer to perform operations, the one or more programs including:

• a first acquisition unit that uses the computer to acquire an operation plan of a piece of equipment, and at least identification information of a parameter among target setting data used for learning of an operation model configured to operate the piece of equipment, the target setting data including identification information of a parameter for which a target range is to be set among parameters relating to the piece of equipment and a target range that is set for a parameter;
• a first learning processing unit that uses the computer to perform, by using learning data including the identification information and the operation plan acquired by the first acquisition unit, learning processing of a target setting model configured to output the target range of the target setting data that is used for learning of the operation model, in response to the operation plan being input wherein the learning processing of the target setting model is performed using machine learning;
• and
• a second learning processing unit that uses the computer to perform, by using learning data including a value of a state parameter relating to the piece of equipment and a value of a control parameter of the piece of equipment and the target setting data, learning processing of the operation model so as to output a value of a control parameter from the operation model causing a state relating to the piece of equipment to be closer to a state corresponding to a content of the target setting data, in response to a value of the state parameter being input;
• wherein the second learning processing unit uses the computer to perform the learning processing of the operation model by reinforcement learning, the learning processing of the operation model including applying a preset reward function that varies a reward value according to a degree that a value of the state parameter relating to the piece of equipment operated using the value of the control parameter outputted from the operation model deviates from the target range of the target setting data.

A non-transitory computer readable medium having recorded thereon one or more programs that cause a computer to perform operations, the one or more programs including:

• a first supply unit that uses the computer to supply, in response to an operation plan of a piece of equipment being newly input, the operation plan to a target setting model configured to output a target range of a parameter among target setting data that is to be used for learning of an operation model configured to operate the piece of equipment, the target setting data including identification information of a parameter for which a target range is to be set among parameters relating to the piece of equipment and the target range that is set for a parameter wherein the target setting model uses machine learning to generate the target range of the target setting data;
• a second acquisition unit that uses the computer to acquire output data from the target setting model supplied with the operation plan by the first supply unit;
• and
• a second learning processing unit that uses the computer to perform learning processing of the operation model configured to operate the piece of equipment by using the target setting data corresponding to the output data acquired by the second acquisition unit;
• wherein the second learning processing unit uses the computer to perform the learning processing of the operation model by reinforcement learning, the learning processing of the operation model including applying a preset reward function that varies a reward value according to a degree that a value of a state parameter relating to the piece of equipment operated using a control parameter output from the operation model deviates from the target range of the target setting data.