Discrete Event Simulation with Sequential Decision Making

A computer-implemented method comprising:

• receiving historical data comprising:
• multiple physical actions taken in an experiment in a physical environment, wherein the experiment comprises a plurality of user-defined stages;
• and
• a recorded outcome, according to one or more user-defined performance indicators related to the plurality of user-defined stages, for each physical action of the multiple physical actions taken in the experiment;
• generating, by a discrete event simulator, a computing representation of a simulated environment of the physical environment, the simulated environment comprising a plurality of processing stages;
• obtaining simulation data comprising:
• multiple simulated actions taken by the discrete event simulator in the simulated environment of the physical environment;
• and
• a predicted outcome, according to one or more user-defined performance indicators related to the plurality of processing stages, for each simulated action of the multiple simulated actions taken by the discrete event simulator;
• validating, based on the historical data, accuracy of the discrete event simulator at predicting the recorded outcome in the experiment;
• and
• training a computing agent according to a sequential decision-making algorithm, by:
• providing, to the computing agent, a computer policy indicating instruction for the computing agent to take an action according to the sequential decision-making algorithm;
• receiving, at the discrete event simulator, from the computing agent, a computer instruction, according to the computer policy, for a simulated action in the discrete event simulator;
• responsive to simulating the simulated action according to the computer instruction, generating a new state in the simulated environment, wherein the new state indicates a predicted result if a physical action is conducted in the physical environment according to the simulated action;
• computing a predicted performance in the simulated environment associated with the new state;
• and
• transmitting, to the computing agent, the new state with a reward indication based on the predicted performance.

A computing system comprising processor and memory, the memory containing instructions executable by the processor wherein the computing system is configured to:

• receive historical data comprising:
• multiple physical actions taken in an experiment in a physical environment, wherein the experiment comprises a plurality of user-defined stages;
• and
• a recorded outcome, according to one or more user-defined performance indicators related to the plurality of user-defined stages, for each physical action of the multiple physical actions taken in the experiment;
• generate, by a discrete event simulator, a computing representation of a simulated environment of the physical environment, the simulated environment comprising a plurality of processing stages;
• obtain simulation data comprising:
• multiple simulated actions taken by the discrete event simulator in the simulated environment of the physical environment;
• and
• a predicted outcome, according to one or more user-defined performance indicators related to the plurality of processing stages, for each simulated action of the multiple simulated actions taken by the discrete event simulator;
• validate, based on the historical data, accuracy of the discrete event simulator at predicting the recorded outcome in the experiment;
• and
• train a computing agent according to a sequential decision-making algorithm, by:
• providing, to the computing agent, a computer policy indicating instruction for the computing agent to take an action according to the sequential decision-making algorithm;
• receiving, at the discrete event simulator, from the computing agent, a computer instruction, according to the computer policy, for a simulated action in the discrete event simulator;
• responsive to simulating the simulated action according to the computer instruction, generating a new state in the simulated environment, wherein the new state indicates a predicted result if a physical action is conducted in the physical environment according to the simulated action;
• computing a predicted performance in the simulated environment associated with the new state;
• and
• transmitting, to the computing agent, the new state with a reward indication based on the predicted performance.