Apparatus, method and storage medium

An apparatus comprising:

• a setting unit for setting an operation content for a manufacturing system configured to manufacture an object to be manufactured, wherein a state parameter set indicates a state of at least one of the manufacturing system or the object to be manufactured and wherein the state parameter set is referred to as a posterior state parameter set when acquired at a time point after the operation content is set by the setting unit,
• a first acquisition unit for acquiring the posterior state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured after the operation content is set, and
• a learning processing unit for executing, by using learning data including the operation content and the posterior state parameter set, a learning process of a control model of the manufacturing system configured to output the operation content that increases a reward value determined by a preset reward function in response to input of the state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured,
• wherein the operation content has a plurality of types of active variables actively settable for the manufacturing system,
• wherein, the learning processing unit is configured to, in a case where a first operation content and a second operation content in which only values of one active variable of the plurality of types of active variables are different to each other are outputted in response to one said state parameter set being input to the control model, and the difference between reward values according to the result of separately setting each of the first operation content and the second operation content in a state indicated by the one state parameter set is less than a reference width, execute the learning process of the control model to output the operation content that does not include the one active variable of the plurality of types of active variables in response to input of the one state parameter set.

An apparatus comprising:

• a setting unit for setting an operation content for a manufacturing system configured to manufacture an object to be manufactured, wherein a state parameter set indicates a state of at least one of the manufacturing system or the object to be manufactured and wherein the state parameter set is referred to as a posterior state parameter set when acquired at a time point after the operation content is set by the setting unit,
• a first acquisition unit for acquiring the posterior state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured after the operation content is set, and
• a learning processing unit for executing, by using learning data including the operation content and the posterior state parameter set, a learning process of a control model of the manufacturing system configured to output the operation content that increases a reward value determined by a preset reward function in response to input of the state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured, further comprising a first storage unit for storing a plurality of variation patterns of the operation content,
• wherein, in a case where a plurality of operation contents are acquired in response to one said state parameter set being input to the control model, the setting unit is configured to set any one of the plurality of operation contents so that each variation pattern in the first storage unit and a variation pattern of the operation content set by the setting unit do not match.

An apparatus comprising:

• a setting unit for setting an operation content for a manufacturing system configured to manufacture an object to be manufactured, wherein a state parameter set indicates a state of at least one of the manufacturing system or the object to be manufactured and wherein the state parameter set is referred to as a posterior state parameter set when acquired at a time point after the operation content is set by the setting unit,
• a first acquisition unit for acquiring the posterior state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured after the operation content is set, and
• a learning processing unit for executing, by using learning data including the operation content and the posterior state parameter set, a learning process of a control model of the manufacturing system configured to output the operation content that increases a reward value determined by a preset reward function in response to input of the state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured, further comprising a first storage unit for storing a plurality of variation patterns of the operation content,
• wherein the reward value, in as case where it is set by the setting unit, is calculated to be lower for the operation content that forms a variation pattern that matches any of the variation patterns in the first storage unit.

A method comprising:

• setting an operation content for a manufacturing system configured to manufacture an object to be manufactured, wherein a state parameter set indicates a state of at least one of the manufacturing system or the object to be manufactured and wherein the state parameter set is referred to as a posterior state parameter set when acquired at a time point after the setting an operation content for a manufacturing system,
• firstly acquiring the posterior state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured after the operation content is set,
• executing, by using learning data including the operation content and the posterior state parameter set, a learning process of a control model of the manufacturing system configured to output the operation content that increases a reward value determined by a preset reward function in response to input of the state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured, and
• executing, in a case where an increase width of the reward value according to a setting result of one said operation content outputted in response to one said state parameter set being input to the control model is less than a reference width, the learning process of the control model not to output the one operation content in response to input of the one state parameter set.

A storage medium having recorded thereon a program for causing a computer to function as:

• a setting unit for setting an operation content for a manufacturing system configured to manufacture an object to be manufactured, wherein a state parameter set indicates a state of at least one of the manufacturing system or the object to be manufactured and wherein the state parameter set is referred to as a posterior state parameter set when acquired at a time point after the operation content is set by the setting unit,
• a first acquisition unit for acquiring the posterior state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured after the operation content is set, and
• a learning processing unit for executing, by using learning data including the operation content and the posterior state parameter set, a learning process of a control model of the manufacturing system configured to output the operation content that increases a reward value determined by a preset reward function in response to input of the state parameter set indicating the state of at least one of the manufacturing system or the object to be manufactured,
• wherein the learning processing unit is configured to, in a case where an increase width of the reward value according to a setting result of one said operation content outputted in response to one said state parameter set being input to the control model is less than a reference width, execute the learning process of the control model not to output the one operation content in response to input of the one state parameter set.