Automated explainer of reinforcement learning actions using occupation measures

A system, comprising:

• at least one hardware processor;
• and
• a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by the at least one hardware processor to:
• receive a reinforcement learning model;
• generate a policy for the reinforcement learning model;
• calculate, based on the generated policy, joint probabilities of a plurality of state-action pairs, wherein each joint probability of the joint probabilities is associated with a respective state-action pair of the plurality of state-action pairs;
• calculate, based on the calculated joint probabilities, a plurality of occupation measures for the plurality of state-action pairs, wherein
• each occupation measure, of the plurality of occupation measures, indicates a frequency of visit of the reinforcement learning model on a respective state-action pair of the plurality of state-action pairs;
• receive automatically, based on a trigger event, an action of interest, wherein the action of interest corresponds to a recommended action, from the reinforcement learning model, to be performed by a system;
• select specific state-action pairs of the plurality of the state-action pairs, wherein
• the specific state-action pairs comprise the action of interest,
• the specific state-action pairs have specific occupation measures, of the plurality of occupation measures, that comply with a predefined threshold, and
• the specific state-action pairs have higher relevance, among the plurality of state-action pairs, based on the specific occupation measures that comply with the predefined threshold;
• identify specific features from the selected specific state-action pairs, wherein the identified specific features correspond to the action of interest;
• reduce a number of the identified specific features based on reduction of dimensionality of feature vectors associated with a corresponding specific state of the specific state-action pairs, wherein
• the reduction of dimensionality of the feature vectors comprises performing principle component analysis (PCA) to identify a number of principle components that corresponds to a desired dimensionality level;
• and
• train, based on the reduced number of the identified specific features, the reinforcement learning model, wherein the training includes updating the policy of the reinforcement learning model with a negative reward for a first state-action pair of the selected specific state-action pairs.

A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by at least one hardware processor to:

• receive a reinforcement learning model;
• generate a policy for the received reinforcement learning model;
• calculate, based on the generated policy, joint probabilities of a plurality of state-action pairs, wherein each joint probability of the joint probabilities is associated with a respective state-action pair of the plurality of state-action pairs;
• calculate, based on the calculated joint probabilities, a plurality of occupation measures for the plurality of state-action pairs, wherein
• each occupation measure, of the plurality of occupation measures, indicates a frequency of visit of the reinforcement learning model on a respective state-action pair of the plurality of state-action pairs;
• receive automatically, based on a trigger event, an action of interest, wherein the action of interest corresponds to a recommended action, from the reinforcement learning model, to be performed by a system;
• select specific state-action pairs of the plurality of state-action pairs, wherein
• the specific state-action pairs comprise the action of interest,
• the specific state-action pairs have specific occupation measures, of the plurality of occupation measures, that comply with a predefined threshold, and
• the specific state-action pairs have higher relevance, among the plurality of state-action pairs, based on the specific occupation measures that comply with the predefined threshold;
• identify specific features from the selected specific state-action pairs, wherein the identified specific features correspond to the action of interest;
• reduce a number of the identified specific features based on reduction of dimensionality of feature vectors associated with a corresponding specific state of the specific state-action pairs, wherein
• the reduction of dimensionality of the feature vectors comprises performing principle component analysis (PCA) to identify a number of principle components that corresponds to a desired dimensionality level;
• and
• train, based on the reduced number of the identified specific features, the reinforcement learning model, wherein the training includes updating the policy of the reinforcement learning model with a lower discount factor for a first state-action pair of the selected specific state-action pairs.