Techniques to set focus in camera in a mixed-reality environment with hand gesture interaction

A method comprising:

• enabling auto-focus operations for a camera in a head-mounted display (HMD) device of a user, the camera being configured to capture a scene in a physical environment surrounding the HMD device over a field of view (FOV);
• controlling, using a location of focus of the user within the FOV, the auto-focus operations for the camera according to a criterion, the criterion indicating that the user is interacting with a virtual object within the FOV;
• and
• rendering content from the HMD device of the user on a computing device of a remote user, the content including the interaction with the virtual object.

A head-mounted display (HMD) device comprising:

• a camera configured to capture a scene in a physical environment surrounding the HMD device over a field of view (FOV);
• a processor;
• and
• a memory storing instructions which, when executed by the processor, cause the HMD device to:
• enable auto-focus operations for the camera;
• detect a location of focus within the FOV;
• and
• controlling, using the detected location of focus, the auto-focus operations for the camera according to a criterion, the criterion indicating that a user is interacting with a virtual object within the FOV.