Query endpointing based on lip detection

A system comprising:

• at least one processor;
• and
• at least one memory storing instructions that, when executed, cause the at least one processor to:
• identify a plurality of training data instances for training a machine learning model, each of the plurality of training data instances comprising:
• training data instance input that includes video data and audio data that is synchronized with the video data, wherein the video data includes at least one video frame, and
• training data instance output that includes an indication of whether the audio data captures a corresponding user speaking in the at least one video frame of the training data instance input;
• and
• train the machine learning model based on the plurality of training data instances to predict whether further video data and further audio data that is synchronized with the further video data captures a user speaking.

A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least at least one processor to:

• identify a plurality of training data instances for training a machine learning model, each of the plurality of training data instances comprising:
• training data instance input that includes video data and audio data that is synchronized with the video data, wherein the video data includes at least one video frame, and
• training data instance output that includes an indication of whether the audio data captures a corresponding user speaking in the at least one video frame of the training data instance input;
• and
• train the machine learning model based on the plurality of training data instances to predict whether further video data and further audio data that is synchronized with the further video data captures a user speaking.