Hand presence over keyboard inclusiveness

A system comprising:

• one or more processors;
• and a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
• access an image of a physical environment of a user, the image being associated with a perspective of the user and depicting a physical input device and a physical hand of the user;
• determine a pose of the physical input device;
• generate a three-dimensional model representing the physical hand of the user;
• generate an image mask by projecting the three-dimensional model onto an image plane associated with the perspective of the user, wherein the image mask comprises a buffer region surrounding a contour of a projection of the three-dimensional model on the image plane;
• generate, by applying the image mask to the image, a cropped image depicting at least the physical hand of the user in the image;
• render, based on the perspective of the user and the pose of the physical input device, a virtual input device to represent the physical input device;
• and
• display the cropped image depicting at least the physical hand of the user over the rendered virtual input device.

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• access an image of a physical environment of a user, the image being associated with a perspective of the user and depicting a physical input device and a physical hand of the user;
• determine a pose of the physical input device;
• generate a three-dimensional model representing the physical hand of the user;
• generate an image mask by projecting the three-dimensional model onto an image plane associated with the perspective of the user, wherein the image mask comprises a buffer region surrounding a contour of a projection of the three-dimensional model on the image plane;
• generate, by applying the image mask to the image, a cropped image depicting at least the physical hand of the user in the image;
• render, based on the perspective of the user and the pose of the physical input device, a virtual input device to represent the physical input device;
• and
• display the cropped image depicting at least the physical hand of the user over the rendered virtual input device.