Interpolation of transition probability values in Markov decision processes

A computer program product for optimizing a control action of a control system using a Markov Decision Process (MDP), by identifying a control action having a highest probability of bringing a respective controlled application system to a system state having an optimal combination of reward and cost values, said computer program product comprising a non-transitory computer readable non-volatile storage medium having stored thereon:

• first program instructions executable by a hardware processor to cause said device to receive a plurality of predefined transition probability values of a Markov Decision Process (MDP) of a control system, wherein each of said plurality of predefined transition probability values is associated with a transition between a first state of a plurality of system states and a second state of said plurality of system states resulting from execution of one of a plurality of control actions of said control system and wherein each of the plurality of system state is associated with at least one reward value and at least one cost value;
• second program instructions executable by said hardware processor to cause said device to compute at least one new transition probability value by an analysis of said plurality of predefined transition probability values, said plurality of system states and said plurality of control actions;
• third program instructions executable by said hardware processor to cause said device to determine at least one recommended control action from said plurality of control actions for respective at least one of said plurality of system states based on said at least one new transition probability value, said at least one recommended control action having a highest probability of bringing a respective application system, controlled by said control system, to a system state associated with the optimal combination of reward and cost values;
• and
• fourth program instructions executable by said hardware processor to cause said device to control said application system by providing said at least one recommended control action for activation by said control system;
• wherein each of said plurality of system states comprises a plurality of system variable values, each of said plurality of system variable values corresponds to a value measured by one of a plurality of sensors of said application system.

A computerized system for optimizing a control action of a control system using a Markov Decision Process (MDP), by identifying a control action having a highest probability of bringing a respective controlled application system to a system state having an optimal combination of reward and cost values, comprising:

• a network interface for receiving system state transition data from at least one of a simulation system and an application system;
• at least one hardware processor;
• and
• at least one memory device having processor instructions stored thereon to cause said at least one hardware processor to perform the actions of:
• receiving a plurality of predefined transition probability values of a Markov Decision Process (MDP) of a control system, wherein each of said plurality of predefined transition probability values is associated with a transition between a first state of a plurality of system states and a second state of said plurality of system states resulting from execution of one of a plurality of control actions of said control system and wherein each of the plurality of system state is associated with at least one reward value and at least one cost value;
• computing at least one new transition probability value by an analysis of said plurality of predefined transition probability values, said plurality of system states and said plurality of control actions;
• determining at least one recommended control action from said plurality of control actions for respective at least one of said plurality of system states based on said at least one new transition probability value, said at least one recommended control action having a highest probability of bringing a respective application system, controlled by said control system, to a system state associated with the optimal combination of reward and cost values;
• and
• controlling said application system by providing said recommended control action for activation by said control system;
• wherein each of said plurality of system states comprises a plurality of system variable values, each of said plurality of system variable values corresponds to a value measured by one of a plurality of sensors of said application system.