Control device, control system, control method, and computer readable medium storing control program

A control system comprising:

• state data acquisition circuitry to acquire state data indicating a state of a control target;
• state category identification circuitry to identify a state category to which a state indicated by the state data belongs among a plurality of state categories indicating classifications of states of the control target on the basis of the state data;
• reward generation circuitry to calculate a reward value of a control detail for the control target on the basis of the state category and the state data;
• control learning circuitry to learn the control detail on the basis of the state data and the reward value;
• training data generation circuitry to generate training data in which the state data and the control detail are associated with each other;
• supervised learning circuitry to generate a supervised learned model for inferring the control detail from the state data on the basis of the training data generated by the training data generation circuitry;
• and
• action inference circuitry to infer the control detail using the supervised learned model,
• wherein
• the reward generation circuitry includes reward calculation formula selection circuitry to select a reward calculation formula different for each of the plurality of state categories on the basis of the inputted state category, and
• reward value calculation circuitry to calculate the reward value using the reward calculation formula selected by the reward calculation formula selection circuitry.

A control method comprising:

• acquiring state data indicating a state of a control target;
• identifying a state category to which a state indicated by the state data belongs among a plurality of state categories indicating classifications of states of the control target on the basis of the state data;
• calculating a reward value of a control detail for the control target on the basis of the state category and the state data;
• and
• learning the control detail on the basis of the state data and the reward value, wherein
• calculating the reward value includes selecting a reward calculation formula different for each of the plurality of state categories on the basis of the inputted state category and calculating the reward value using the selected reward calculation formula.

A non-transitory computer-readable storage medium storing a program for causing a computer to execute processes of:

• acquiring state data indicating a state of a control target;
• identifying a state category to which a state indicated by the state data belongs among a plurality of state categories indicating classifications of states of the control target on the basis of the state data;
• calculating a reward value of a control detail for the control target on the basis of the state category and the state data;
• and
• learning the control detail on the basis of the state data and the reward value,
• wherein the process of calculating the reward value includes a process of selecting a reward calculation formula different for each of the plurality of state categories on the basis of the inputted state category and a process of calculating the reward value using the selected reward calculation formula.