Multi-objective customer journey optimization

A system comprising:

• a database storing user information for a plurality of users, the user information including user interaction data indicating an interest level of a user at a point in a user journey;
• a message generation component configured to generate messages having a message type selected from a plurality of message types, wherein the plurality of message types correspond to interest levels representing the user journey;
• a decision making component comprising a neural network that is trained based on user responses to the plurality of message types, wherein the neural network represents a policy function of a Markov Decision Process (MDP) model, and wherein an input to the neural network includes a state variable based on the user information, and an output of the neural network comprises a probability vector including a plurality of values corresponding to the plurality of message types, respectively;
• a message transmission component configured to transmit a message to a user among the plurality of users determined by the decision making component;
• and
• a data collection component configured to identify a user interaction in response to the transmitted message, update the user information based on the user interaction, and determine whether to transmit a subsequent message during a subsequent time period based on the updated user information, wherein:
• the decision making component is further configured to:
• identify a user subgroup for new behavior exploration;
• determine that the customer belongs to the user subgroup;
• identify a probability for a random delivery schedule based on the determination;
• and
• identify a random message based at least in part on the probability;
• and wherein the message transmission component is further configured to:
• transmit the random message to the customer;
• identify a result of the random message;
• and
• update the neural network, the user information, or both based on the result.

A method of training a neural network, the method comprising:

• training a deep Q-learning neural network representing a policy function of a Markov Decision Process (MDP) model comprising an action set including times of day and a plurality of message types corresponding to a customer journey, wherein the deep Q-learning neural network is trained based on customer responses to the plurality of message types;
• identifying user information for a customer, wherein the user information includes user interaction data;
• determining a message type from the plurality of message types and a time of day using the deep Q-learning neural network and the user information;
• selecting a message based on the determined message type;
• transmitting the message to the customer at the determined time of day based on the selection;
• identifying a customer interaction in response to the message;
• updating the deep Q-learning neural network based on the customer interaction;
• and
• exploring additional subgroups for new behavior, wherein the exploring additional subgroups comprises:
• identifying a user subgroup for new behavior exploration;
• determining that the customer belongs to the user subgroup;
• identifying a probability for a random delivery schedule based on the determination;
• identifying a random message based at least in part on the probability;
• transmitting the random message to the customer;
• identifying a result of the random message;
• and
• updating the neural network, the user information, or both based on the result.