In-call experience enhancement for assistant systems

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• establish a video call between a plurality of client systems, wherein access to an assistant system is persistently maintained during the video call;
• receive, from a first client system of the plurality of client systems, a request by a first user to be performed by the assistant system during the video call, wherein the request references one or more activities to be performed after the request and is associated with one or more users associated with the plurality of client systems, wherein the one or more activities are physical actions performed by the one or more users after the request is made;
• analyze, by a context engine of the assistant system after receiving the request from the first user, images of a scene of the video call to identify within the scene the one or more activities referenced in the request;
• instruct the assistant system to execute the request responsive to the identification of one or more of the activities referenced in the request;
• and
• send, to one or more of the plurality of client systems, a response to the request while maintaining the video call between the plurality of client systems.

A system comprising one or more processors and one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable, when executed by one or more of the processors, to cause the system to:

• establish a video call between a plurality of client systems, wherein access to an assistant system is persistently maintained during the video call;
• receive, from a first client system of the plurality of client systems, a request by a first user to be performed by the assistant system during the video call, wherein the request references one or more activities to be performed after the request and is associated with one or more users associated with the plurality of client systems, wherein the one or more activities referenced in the request are physical actions performed by the one or more users after the request is made;
• analyze, by a context engine of the assistant system after receiving the request from the first user, images of a scene of the video call to identify within the scene the one or more activities referenced in the request;
• instruct the assistant system to execute the request responsive to the identification of one or more of the activities referenced in the request;
• and
• send, to one or more of the plurality of client systems, a response to the request while maintaining the video call between the plurality of client systems.