Artificial reality system with varifocal display of artificial reality content

A method comprising:

• capturing, by a first image capture device, real-world image data representative of a physical environment of a user;
• capturing, by a set of second image capture devices of a head-mounted display (HMD) configured to output artificial reality content, image data indicative of a focal point of a gaze of the user;
• modifying, by a depth engine of the HMD, a focal length of a varifocal display of the HMD based on the focal point of the user;
• generating, by the depth engine and based on the real-world image data and depth data associated with the real-world image data, a three-dimensional (3D) scene of the physical environment of the user;
• and
• generating, by the depth engine, artificial reality content as an overlay to the 3D scene of the physical environment for display on the varifocal display of the HMD based on the focal point of the user.

A non-transitory computer-readable medium comprising instructions that, when executed, cause one or more processors to:

• capture real-world image data representative of a physical environment of a user;
• capture image data indicative of a focal point of a gaze of the user;
• modify a focal length of a varifocal display based on the focal point of the user;
• generate, based on the real-world image data and depth data associated with the real-world image data, a three-dimensional (3D) scene of the physical environment of the user;
• and
• generate artificial reality content as an overlay to the 3D scene of the physical environment for display on the varifocal display of a head-mounted display (HMD) based on the focal point of the user.