Optimizing a machine for solving offline optimization problems

A system for improving a machine operation, the system comprising:

• one or more computer readable storage media with program instructions collectively stored on the one or more computer readable storage media;
• and
• one or more processors configured to execute the program instructions to perform a method comprising:
• receiving a plurality of domain specific heuristics, a set of states, a set of actions, where an immediate cost and/or reward is associated with a pair of state and action;
• generating at least one of:
• a graph of state transitions for the actions, and
• a transition probability matrix;
• executing a Markov Decision Process (MDP) model for solving an MDP problem, and outputting an MDP optimal policy of an optimal mapping of a given state to an action;
• selecting one of the plurality of domain specific heuristics and heuristic input parameters thereof;
• controlling the machine for solving a predefined optimization problem in a plurality of execution iterations, wherein each execution iteration includes:
• using the outputted MDP action for a current state which may include the selected domain specific heuristic and its input parameters;
• receiving a result for the optimization problem and calculating a next state;
• upon the MDP model determining that a predefined stopping condition is met, stopping the execution iterations of the machine for solving the predefined optimization problem;
• upon the MDP model determining that the predefined stopping has not been met, inputting the next state to the MDP model to receive an optimal action for the next state for a next iteration of the machine for solving the predefined optimization problem.

A computer program product for improving a machine operation, the computer program product comprising:

• one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions comprising:
• program instructions to receive a plurality of domain specific heuristics, a set of states, a set of actions, where an immediate cost and/or reward is associated with a pair of state and action;
• program instructions to generate at least one of:
• a graph of state transitions for the actions, and
• a transition probability matrix;
• program instructions to execute a Markov Decision Process (MDP) model for solving an MDP problem, and outputting an MDP optimal policy of an optimal mapping of a given state to an action;
• program instructions to select one of the plurality of domain specific heuristics and heuristic input parameters thereof based on outcomes of a previous iteration;
• program instructions to control the machine for solving a predefined optimization problem in a plurality of execution iterations, wherein each execution iteration includes:
• using the outputted MDP action for a current state which may include the selected domain specific heuristic and its input parameters;
• receiving a result for the optimization problem and calculating a next state;
• upon the MDP model determining that a predefined stopping condition is met, stopping the execution iterations of the machine for solving the predefined optimization problem;
• upon the MDP model determining that the predefined stopping has not been met, inputting the next state to the MDP model to receive an optimal action for the next state for a next iteration of the machine for solving the predefined optimization problem.