Reward-based recommendations of actions using machine-learning on telemetry data

A non-transitory computer-readable storage medium, containing a set of instructions which, when executed by a processor, cause the processor to perform a method for automatically providing action recommendations in a cloud environment including one or more physical machines, one or more virtual machines, and a database, the method comprising:

• collecting, by an action recommendation manager implemented in the cloud environment, a set of telemetry data from a client application operating in the cloud environment and storing the set of telemetry data in the database, wherein the set of telemetry data contains a plurality of pages generated based on a plurality of user actions performed on the client application;
• generating, by the action recommendation manager, a plurality of prior probabilities corresponding to the plurality of pages and the plurality of user actions;
• in response to the client application displaying a first page selected from the plurality of pages, generating, by the action recommendation manager, a plurality of posterior probabilities for a subset of user actions selected from the plurality of user actions based on the plurality of prior probabilities;
• selecting, by the action recommendation manager, a first plurality of recommended actions from the subset of user actions for having the highest corresponding posterior probabilities among the plurality of posterior probabilities;
• generating, by the action recommendation manager, a machine-learning matrix based on the first plurality of recommended actions and a first performed user action from the client application and storing the machine-learning matrix in the database, wherein the machine-learning matrix contains associations among the plurality of pages and the plurality of user actions and preference values for the associations, and a machine-learning module in the action recommendation manager iteratively adjusts the preference values in the machine-learning matrix based on an action-value function until changes of the preference values are within a threshold;
• and
• in response to the client application displaying, a second page selected from the plurality of pages, generating, by the action recommendation manager, a second plurality of recommended actions based on the machine-learning matrix, wherein the second plurality of recommended actions have the highest preference values based on the second page.

A system for automatically providing action recommendations in a cloud environment including one or more physical machines, one or more virtual machines, and a database, the system comprising:

• a telemetry-data-collector implemented in the cloud environment and configured to collect a set of telemetry data from a client application operating in the cloud environment and store the set of telemetry data in the database, wherein the set of telemetry data contains a plurality of pages generated based on a plurality of user actions performed on the client application;
• and
• a recommendation-module implemented in the cloud environment and coupled with the telemetry-data-collector, wherein the recommendation-module is configured to
• generate a plurality of prior probabilities corresponding to the plurality of pages and the plurality of user actions,
• in response to the client application displaying a first page selected from the plurality of pages, generate a plurality of posterior probabilities for a subset of user actions selected from the plurality of user actions and can be invoked in the client application from the first page, wherein the plurality of posterior probabilities are generated based on the plurality of prior probabilities, and
• select a first plurality of recommended actions from the subset of user actions for having the highest corresponding posterior probabilities among the plurality of posterior probabilities;
• a machine-learning module implemented in the cloud environment, wherein the machine-learning module is configured to
• generate a machine-learning matrix based on the first plurality of recommended actions and a first performed user action from the client application and store the machine-learning matrix in the database, wherein the machine-learning matrix contains associations among the plurality of pages and the plurality of user actions and preference values for the associations, and;
• iteratively adjust the preference values in the machine-learning matrix based on an action-value function until changes of the preference values are within a threshold;
• and
• wherein the recommendation-module is further configured to in response to the client application displaying a second page selected from the plurality of pages, generate a second plurality of recommended actions based on the machine-learning matrix, wherein the second plurality of recommended actions have the highest preference values based on the second page.