# US 12,008,763 B2 · Variable aperture (VA) camera control for controlling a depth of focus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,763 B2 - Document ID: US12008763B2 - Status: Granted - Kind: B2 - Application number: 17/817,584 - Filed: 2022-08-04 - Published: 2024-02-08 - Granted: 2024-06-11 - Earliest claimed: 2022-08-04 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Wen-Chun Feng; Micha Galor Gluskin; Hui Shan Kao; Hang-Wei Liaw - Web page: https://patentgrep.com/patent/US12008763B2 ## CPC - G06T 7/11: Region-based segmentation - G03B 9/00: Exposure-making shutters; Diaphragms - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - H04N 5/2226: {Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10)} - H04N 23/611: where the recognised objects include parts of the human body - H04N 23/67: Focus control based on electronic image sensor signals - H04N 23/72: Combination of two or more compensation controls - H04N 23/959: by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics - G03B 13/36: Autofocus systems - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract This disclosure provides systems, methods, and devices for image capture and image processing that support capture using a variable aperture (VA) camera. In a first aspect, a method of image processing includes determining a plurality of depths for a scene, the plurality of depths corresponding to a plurality of regions of interest; determining an aperture size for a camera based on the plurality of depths; determining a lens position for the camera based on the aperture size and the plurality of depths; and controlling the camera to adjust a variable aperture (VA) of the camera based on the aperture size and to adjust a lens associated with the variable aperture (VA) based on the lens position. Other aspects and features are also claimed and described. ## Claims ### Claim 1 A method, comprising: determining a plurality of depths for a scene and a plurality of regions of interest, wherein a first one of the plurality of regions of interest corresponds to a first one of the plurality of depths, which is at a first distance from a reference point, and a second one of the plurality of regions of interest corresponds to a second one of the plurality of depths, which is at a second distance from the reference point; determining an aperture size for a camera based on a difference between the first distance and the second distance; determining a lens position for the camera that provides a focal plane that is between the first distance and the second distance based on the aperture size and the first one of the plurality of depths and the second one of the plurality of depths; and controlling the camera to adjust a variable aperture (VA) of the camera based on the determined aperture size and to adjust a lens associated with the variable aperture (VA) based on the determined lens position. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the aperture size comprises determining an aperture size having a depth of focus at least as large as the difference between the first distance and the second distance. ### Claim 3 (depends on 2) The method of claim 2, wherein determining the lens position comprises determining a lens position such that the first one of the plurality of depths and the second one of the plurality of depths are both within the depth of focus extending around a focal plane of the lens position. ### Claim 4 (depends on 1) The method of claim 1, wherein determining the plurality of regions of interest comprises determining locations of at least two faces in the scene, wherein the first of the plurality of depths and the second of the plurality of depths correspond to respective depths at the locations of the at least two faces. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the aperture size for the camera comprises: determining an initial aperture size based on light conditions in the scene; and adjusting the initial aperture size to determine the aperture size based on the difference between the first distance and the second distance. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the plurality of depths for a scene comprises receiving depth data for the scene from a depth sensor. ### Claim 7 (depends on 1) The method of claim 1, further comprising: receiving image data from the camera, the image data captured at the variable aperture and the lens position; and determining at least one image frame based on the image data. ### Claim 8 (depends on 7) The method of claim 7, further comprising: determining a camera depth of focus for the camera at the lens position and at the aperture size is less than a distance between a first depth of the plurality of depths and a second depth of the plurality of depths, wherein the determining the at least one image frame comprises deblurring at least a portion of one region of interest of the plurality of regions of interest that is beyond the camera depth of focus. ### Claim 9 (depends on 1) The method of claim 1, further comprising: receiving first image data representing the scene from the camera, wherein the determining of the plurality of regions of interest comprises determining a plurality of locations of a plurality of objects of the scene by performing object detection on the first image data. ### Claim 10 (depends on 9) The method of claim 9, further comprising: receiving user input to specify the plurality of regions of interest from the plurality of locations, wherein the first one of the plurality of depths and the second one of the plurality of depths correspond to objects in the scene selected by a user. ### Claim 11 (depends on 10) The method of claim 10, wherein determining the plurality of locations of a plurality of objects of the scene comprises determining a plurality of locations of a plurality of faces of the scene by performing facial recognition on the first image data. ### Claim 12 (depends on 10) The method of claim 10, wherein determining the plurality of locations of a plurality of objects of the scene comprises determining a plurality of locations by applying a saliency model to the first image data to determine locations having a salience score above a threshold value. ### Claim 13 (depends on 10) The method of claim 10, further comprising: receiving second image data from the camera after controlling the camera to adjust a variable aperture (VA) of the camera based on the aperture size and to adjust a lens associated with the variable aperture (VA) based on the lens position, the second image data captured based on the aperture size and the lens position; and determining at least one image frame based on the second image data. ### Claim 14 An apparatus, comprising: a memory storing processor-readable code; and at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including: determining a plurality of depths for a scene and a plurality of regions of interest, wherein a first one of the plurality of regions of interest corresponds to a first one of the plurality of depths, which is at a first distance from a reference point, and a second one of the plurality of regions of interest corresponds to a second one of the plurality of depths, which is at a second distance from the reference point; determining an aperture size for a camera based on a difference between the first distance and the second distance; determining a lens position for the camera that provides a focal plane that is between the first distance and the second distance based on the aperture size and the first one of the plurality of depths and the second one of the plurality of depths; and controlling the camera to adjust a variable aperture (VA) of the camera based on the determined aperture size and to adjust a lens associated with the variable aperture (VA) based on the determined lens position. ### Claim 15 (depends on 14) The apparatus of claim 14, wherein determining the lens position comprises determining a lens position such that the first one of the plurality of depths and the second one of the plurality of depths are both within the depth of focus extending around a focal plane of the lens position. ### Claim 16 (depends on 14) The apparatus of claim 14, wherein determining the aperture size comprises determining an aperture size having a depth of focus at least as large as the different between the first distance and the second distance. ### Claim 17 (depends on 14) The apparatus of claim 14, wherein determining the plurality of regions of interest comprises determining locations of at least two faces in the scene, wherein the first of the plurality of depths and the second of the plurality of depths correspond to respective depths at the locations of the at least two faces. ### Claim 18 (depends on 14) The apparatus of claim 14, wherein determining the aperture size for the camera comprises: determining an initial aperture size based on light conditions in the scene; and adjusting the initial aperture size to determine the aperture size based on the difference between the first distance and the second distance. ### Claim 19 (depends on 14) The apparatus of claim 14, wherein determining the plurality of depths for a scene comprises receiving a depth map for the scene from a depth sensor. ### Claim 20 (depends on 14) The apparatus of claim 14, wherein the processor is further configured to perform operations comprising: receiving image data from the camera, the image data captured at the variable aperture and the lens position; and determining at least one image frame based on the image data. ### Claim 21 (depends on 20) The apparatus of claim 20, wherein the processor is further configured to perform operations comprising: determining a camera depth of focus for the camera at the lens position and at the aperture size is less than a distance between a first depth of the plurality of depths and a second depth of the plurality of depths, wherein the determining the at least one image frame comprises deblurring at least a portion of one region of interest of the plurality of regions of interest that is beyond the camera depth of focus. ### Claim 22 (depends on 14) The apparatus of claim 14, wherein the processor is further configured to perform operations comprising: receiving first image data representing the scene from the camera; and determining a plurality of locations of a plurality of objects of the scene by performing object detection on the first image data, wherein the plurality of depths correspond to the plurality of objects. ### Claim 23 (depends on 22) The apparatus of claim 22, wherein the processor is further configured to perform operations comprising: receiving user input to specify the plurality of regions of interest from the plurality of locations, wherein the first one of the plurality of depths and the second one of the plurality of depths correspond to objects in the scene selected by a user. ### Claim 24 (depends on 23) The apparatus of claim 23, wherein determining the plurality of locations of a plurality of objects of the scene comprises determining a plurality of locations of a plurality of faces of the scene by performing facial recognition on the first image data. ### Claim 25 (depends on 23) The apparatus of claim 23, wherein determining the plurality of locations of a plurality of objects of the scene comprises determining a plurality of locations by applying a saliency model to the first image data to determine locations having a salience score above a threshold value.