# US 12,003,691 B1 · Camera focus adjustment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,003,691 B1 - Document ID: US12003691B1 - Status: Granted - Kind: B1 - Application number: 18/310,825 - Filed: 2023-05-02 - Granted: 2024-06-04 - Earliest claimed: 2023-05-02 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Minseok Oh; Rui Jin - Web page: https://patentgrep.com/patent/US12003691B1 ## CPC - H04N 13/128: Adjusting depth or disparity - H04N 23/675: {comprising setting of focusing regions} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - H04N 5/2226: {Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10)} - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 23/20: for generating image signals from infrared radiation only - H04N 23/56: provided with illuminating means - H04N 23/611: where the recognised objects include parts of the human body - H04N 23/67: Focus control based on electronic image sensor signals - H04N 23/74: by influencing the scene brightness using illuminating means - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract A method for camera focus adjustment includes, at a computing system, receiving a depth image of a surrounding environment captured by a depth camera of the computing system. The depth image includes a plurality of depth pixels each encoding depth values corresponding to distances between the depth camera and objects in the surrounding environment. One or more depth pixels of the plurality of depth pixels are identified as a region of interest (ROI). Based on the depth values of the one or more depth pixels, a focus of an environmental imaging camera of the computing system is adjusted. ## Claims ### Claim 1 A method for camera focus adjustment, the method comprising: at a computing system, receiving a depth image of a surrounding environment captured by a depth camera of the computing system, the depth image including a plurality of depth pixels, and the plurality of depth pixels each encoding depth values corresponding to distances between the depth camera and objects in the surrounding environment; identifying one or more depth pixels of the plurality of depth pixels as a region-of-interest (ROI); and adjusting a focus of an environmental imaging camera of the computing system based at least in part on the depth values of the one or more depth pixels. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more depth pixels are identified as the ROI based at least in part on determining that the one or more depth pixels correspond to a gaze vector of a human user, the gaze vector received from a gaze detection system of the computing system. ### Claim 3 (depends on 1) The method of claim 1, further comprising receiving an environmental image of the surrounding environment captured by the environmental imaging camera, and detecting an object of interest in the environmental image, and wherein the one or more depth pixels are identified as the ROI based at least in part on determining that a pixel-space position of the object of interest in the environmental image corresponds to the one or more depth pixels in the depth image. ### Claim 4 (depends on 1) The method of claim 1, further comprising, after adjusting the focus of the environmental imaging camera, receiving an environmental image of the surrounding environment captured by the environmental imaging camera, and displaying the environmental image. ### Claim 5 (depends on 1) The method of claim 1, further comprising, after identifying the one or more depth pixels as the ROI, adjusting an active illumination emitted by the depth camera to reduce active illumination of objects in the surrounding environment not imaged by the one or more depth pixels, while still actively illuminating an object in the surrounding environment imaged by the one or more depth pixels. ### Claim 6 (depends on 1) The method of claim 1, further comprising, after identifying the one or more depth pixels as the ROI, adjusting an active illumination emitted by the depth camera to increase active illumination of an object in the surrounding environment imaged by the one or more depth pixels. ### Claim 7 (depends on 1) The method of claim 1, further comprising identifying a second one or more depth pixels of the plurality of depth pixels as a second ROI, and adjusting the focus of the environmental imaging camera based at least on in part on the depth values of the second one or more depth pixels. ### Claim 8 (depends on 7) The method of claim 7, wherein the focus of the environmental imaging camera is set to an intermediary focus level corresponding to a distance range that is between the depth values of the one or more depth pixels and the depth values of the second one or more depth pixels. ### Claim 9 (depends on 7) The method of claim 7, further comprising displaying an environmental image of the surrounding environment captured by the environmental imaging camera, and superimposing a first bounding box and a second bounding box over pixels of the environmental image corresponding to the ROI and the second ROI. ### Claim 10 (depends on 9) The method of claim 9, further comprising receiving a user input specifying either the ROI or the second ROI, and wherein the focus of the environmental imaging camera is set to either a first focus level for the ROI or a second focus level for the second ROI according to the user input. ### Claim 11 (depends on 7) The method of claim 7, further comprising receiving an environmental image of the surrounding environment captured by the environmental imaging camera while the environmental imaging camera is set to a first focus level for the ROI, deblurring a portion of the environmental image corresponding to the second ROI based at least in part on the depth values of the second one or more depth pixels and a point spread function of the environmental imaging camera, and displaying the environmental image. ### Claim 12 (depends on 7) The method of claim 7, further comprising receiving a first environmental image of the surrounding environment captured by the environmental imaging camera while the environmental imaging camera is set to a first focus level for the ROI, receiving a second environmental image of the surrounding environment captured by the environmental imaging camera while the environmental imaging camera is set to a second focus level for the second ROI, deriving a hybrid environmental image from the first environmental image and the second environmental image, and displaying the hybrid environmental image. ### Claim 13 (depends on 1) The method of claim 1, wherein the focus of the environmental imaging camera is adjusted based on a preexisting camera calibration that maps different distance ranges to different focus levels. ### Claim 14 (depends on 1) The method of claim 1, wherein the environmental imaging camera is a visible light camera configured for use in low-light environments. ### Claim 15 (depends on 1) The method of claim 1, wherein the environmental imaging camera is an infrared camera. ### Claim 16 (depends on 1) The method of claim 1, wherein the depth camera is a short-wave infrared (SWIR) depth camera. ### Claim 17 A head-mounted display device (HMD), comprising: an environmental imaging camera; a depth camera; a logic subsystem; and a storage subsystem holding instructions executable by the logic subsystem to: receive a depth image of a surrounding environment captured by the depth camera, the depth image including a plurality of depth pixels, and the plurality of depth pixels each encoding depth values corresponding to distances between the depth camera and objects in the surrounding environment; identify one or more depth pixels of the plurality of depth pixels as a region-of-interest (ROI); and adjust a focus of the environmental imaging camera based at least in part on the depth values of the one or more depth pixels. ### Claim 18 (depends on 17) The HMD of claim 17, wherein the one or more depth pixels are identified as the ROI based at least in part on determining that the one or more depth pixels correspond to a gaze vector of a human user, the gaze vector received from a gaze detection system of the HMD. ### Claim 19 (depends on 17) The HMD of claim 17, wherein the instructions are further executable to receive an environmental image of the surrounding environment captured by the environmental imaging camera, and detect an object of interest in the environmental image, and wherein the one or more depth pixels are identified as the ROI based at least in part on determining that a pixel-space position of the object of interest in the environmental image corresponds to the one or more depth pixels in the depth image. ### Claim 20 A method for camera focus adjustment, the method comprising: at a head-mounted display device (HMD), receiving an environmental image of a surrounding environment captured by an environmental imaging camera of the HMD; detecting an object of interest at a pixel-space position in the environmental image; receiving a depth image of the surrounding environment captured by a depth camera of the HMD, the depth image including a plurality of depth pixels, and the plurality of depth pixels each encoding depth values corresponding to distances between the depth camera and objects in the surrounding environment; identifying one or more depth pixels of the plurality of depth pixels as a region-of-interest based at least in part on determining that the pixel-space position of the object of interest in the environmental image corresponds to the one or more depth pixels in the depth image; and adjusting a focus of the environmental imaging camera based at least in part on the depth values of the one or more depth pixels.