Reinforcement learning system for maintenance decision making

A non-transitory computer readable medium, storing instructions for predictive maintenance of equipment, the instructions, when executed by a processor, performing operations comprising:

• receiving an expected future return value as input to a decision maker model, wherein the decision maker model is a machine learning model that predicts a maintenance action associated with the equipment;
• feeding recent observations and recent actions from an environment as inputs to the decision maker model;
• generating a next action as model outputs of the decision maker model, wherein the next action is the predicted maintenance action associated with the equipment, a confidence score indicating reliability of the next action, and explanation information comprising feature-wise explanation identifying which input features are most responsible for the next action and instance-wise explanation showing which training samples from an offline reinforcement learning dataset comprising past maintenance observations, actions, and rewards most influenced the predicted maintenance action;
• and
• executing the next action in the environment.

An offline reinforcement learning method for predictive maintenance of equipment, the method comprising:

• preparing time-series data of past observations and associated past actions;
• splitting the time-series data into episodes and computing rewards associated with the episodes;
• storing the episodes and the rewards in a database;
• initializing a decision maker model, wherein the decision maker model is a machine learning model that predicts a maintenance action associated with the equipment;
• training the decision maker model by randomly drawing a sample batch from the database, computing loss associated with the sample batch, and updating parameters of the decision maker model using gradient descent of the loss;
• receiving an expected future return value as input to the decision maker model;
• feeding recent observations and recent actions from an environment as inputs to the decision maker model;
• generating a next action as model outputs of the decision maker model, wherein the next action is the predicted maintenance action associated with the equipment, a confidence score indicating reliability of the next action, and explanation information comprising feature-wise explanation identifying which input features are most responsible for the next action and instance-wise explanation showing which training samples from an offline reinforcement learning dataset comprising past maintenance observations, actions, and rewards most influenced the predicted maintenance action;
• and
• executing the next action in the environment.