Generating smart reminders by assistant systems

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• receive, at the mobile client system, initial sensory data captured by one or more cameras of the mobile client system, wherein the initial sensory data is visual data;
• proactively generate, by the mobile client system responsive to proactively identifying a first entity based on a visual analysis of the visual data and correlating the first entity with knowledge about the user, a reminder associated with the first entity for the user, wherein the knowledge about the user is obtained by analyzing one or more of a routine of the user related to the first entity or an episodic memory of the user referencing the first entity;
• determine, by the mobile client system, an activation condition associated with the reminder, wherein the activation condition is based on one or more of a time or a location, wherein the time and/or location are determined based on the analysis of the visual data and the knowledge about the user;
• store, at a data store of the mobile client system, an intent associated with the reminder in an intent stack on the data store based on a priority of the intent;
• determine, by the mobile client system, the priority of the intent meets a criteria;
• determine, by the mobile client system responsive to determining the priority of the intent meets the criteria, whether the activation condition is satisfied;
• and
• present, at the mobile client system, the reminder when the activation condition is satisfied, wherein the reminder comprises an activatable prompt for a user to execute a task referenced in the reminder.

A system comprising:

• one or more processors;
• and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
• receive, at the mobile client system, initial sensory data captured by one or more cameras of the mobile client system, wherein the initial sensory data is visual data;
• proactively generate, by the mobile client system responsive to proactively identifying a first entity based on a visual analysis of the visual data and correlating the first entity with knowledge about the user, a reminder associated with the first entity for the user, wherein the knowledge about the user is obtained by analyzing one or more of a routine of the user related to the first entity or an episodic memory of the user referencing the first entity;
• determine, by the mobile client system, an activation condition associated with the reminder, wherein the activation condition is based on one or more of a time or a location, wherein the time and/or location are determined based on the analysis of the visual data and the knowledge about the user;
• store, at a data store of the mobile client system, an intent associated with the reminder in an intent stack on the data store based on a priority of the intent;
• determine, by the mobile client system, the priority of the intent meets a criteria;
• determine, by the mobile client system responsive to determining the priority of the intent meets the criteria, whether the activation condition is satisfied;
• and
• present, at the mobile client system, the reminder when the activation condition is satisfied, wherein the reminder comprises an activatable prompt for a user to execute a task referenced in the reminder.