Learning: learning design policies based on interactions

A non-transitory computer-readable storage medium storing instructions that when executed by a processor of a computing device, cause the processor to:

• determine a first task, of a plurality of tasks, for an environment, the environment comprising one or more of a computing environment and a real-world environment, the first task relative to one or more of the computing environment and the real-world environment;
• receive data from a plurality of data sources, the received data comprising audio data, image data, application data, and text data;
• extract a plurality of features from the received data, the plurality of features comprising:
• objects identified in the image data, natural language concepts extracted from the audio data, and natural language concepts extracted from the text data;
• determine a first time step associated with the received data;
• determine a plurality of candidate actions for the determined first time step;
• compute a respective first probability value of each candidate action achieving the first task at the first time step based on a first machine learning (ML) model applied to the received data and the plurality of features, the first ML model trained based on training data labeled to specify that the training data achieves each of the plurality of tasks or the training data does not achieve each of the plurality of tasks;
• determine that a first candidate action of the plurality of candidate actions has a greater first probability value for achieving the first task at the first time step relative to the first probability values of the remaining plurality of candidate actions;
• determine that the first candidate action has not been implemented in the environment at the first time step;
• compute, based on the first ML model, a second probability value for the first candidate action, the second probability value reflecting a likelihood that the first candidate action will occur in the real-world environment at the first time step;
• generate, based on the first and second probability values of the first candidate action, an indication specifying to implement the first candidate action at the first time step as part of a policy to achieve the first task;
• generate, by a second ML model based on the first ML model, the plurality of candidate actions, the probability values, the received data from the plurality of data sources, and the plurality of features, a natural language narrative describing the first candidate action and the second probability value for the first candidate action, the natural language narrative comprising a plurality of graphical symbols associated with a plurality of natural language concepts of the natural language narrative, the plurality of natural language concepts of the natural language narrative comprising at least one of the natural language concepts extracted from the audio data and at least one of the natural language concepts extracted from the text data;
• compute, based on the first ML model, a review probability value reflecting whether a user will submit a review of an application in the computing environment;
• and
• update the training data based on the review probability value.

A method, comprising:

• determining a first task, of a plurality of tasks, for an environment, the environment comprising one or more of a computing environment and a real-world environment, the first task relative to one or more of the computing environment and the real-world environment;
• receiving data from a plurality of data sources, the received data comprising audio data, image data, application data, and text data;
• extracting a plurality of features from the received data, the plurality of features comprising:
• objects identified in the image data, natural language concepts extracted from the audio data, and natural language concepts extracted from the text data;
• determining a first time step associated with the received data;
• determining a plurality of candidate actions for the determined first time step;
• computing a respective first probability value of each candidate action achieving the first task at the first time step based on a first machine learning (ML) model applied to the received data and the plurality of features, the first ML model trained based on training data labeled to specify that the corresponding training data achieves each of the plurality of tasks or the training data does not achieve each of the plurality of tasks;
• determining that a first candidate action of the plurality of candidate actions has a greater first probability value for achieving the first task at the first time step relative to the first probability values of the remaining plurality of candidate actions;
• determining that the first candidate action has not been implemented in the environment at the first time step;
• computing, based on the first ML model, a second probability value for the first candidate action, the second probability value reflecting a likelihood that the first candidate action will occur in the real-world environment at the first time step;
• generating, based on the first and second probability values of the first candidate action, an indication specifying to implement the first candidate action at the first time step as part of a policy to achieve the first task;
• generating, by a second ML model and based on the first ML model, the first candidate action, the second probability value, the received data from the plurality of data sources, and the plurality of features, a natural language narrative describing the first candidate action and the second probability value for the first candidate action, the natural language narrative comprising a plurality of graphical symbols associated with a plurality of natural language concepts of the natural language narrative, the plurality of natural language concepts of the natural language narrative comprising at least one of the natural language concepts extracted from the audio data and at least one of the natural language concepts extracted from the text data;
• computing, based on the first ML model, a review probability value reflecting whether a user will submit a review of an application in the computing environment;
• and
• updating the training data based on the review probability value.