# US 12,192,623 B2 · Depth sensing techniques for virtual, augmented, and mixed reality systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,623 B2 - Document ID: US12192623B2 - Status: Granted - Kind: B2 - Application number: 18/453,762 - Filed: 2023-08-22 - Published: 2023-12-07 - Granted: 2025-01-07 - Earliest claimed: 2017-03-21 - Assignees: Magic Leap, Inc. - Applicants: Magic Leap, Inc. - Inventors: Brian Keith Smith; Koon Keong Shee; Gregory Michael Link - Web page: https://patentgrep.com/patent/US12192623B2 ## CPC - G06F 3/012: {Head tracking input arrangements} - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar - G06F 3/00: Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements - H04N 13/139: Format conversion, e.g. of frame-rate or size - H04N 13/296: Synchronisation thereof; Control thereof - H04N 23/959: by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract A system and method for operating a depth sensor. A configuration operation can be performed by storing a first sequence of operation steps which define a first depth sensing mode of operation, and a second sequence of operation steps which define a second depth sensing mode of operation, in the memory. In response to a first request for depth measurements according to the first depth sensing mode of operation, the depth sensor can be operated in the first mode of operation by causing it to execute the first sequence of operation steps. In response to a second request for depth measurements according to the second depth sensing mode of operation, and without performing an additional configuration operation, the depth sensor can be operated in the second mode of operation by causing it to execute the second sequence of operation steps. ## Claims ### Claim 1 A depth sensor comprising: memory storing information usable to operate the depth sensor, the information including: a first sequence of operation steps which define a first depth sensing mode of operation of the depth sensor, the first sequence of operation steps including one or more common operation steps, and a second sequence of operation steps which, in combination with the one or more common operation steps, define a second depth sensing mode of operation of the depth sensor; and a processor communicatively coupled to the memory, the processor configured to: receive a first request for first depth measurements according to the first depth sensing mode of operation and, in response to the first request, operate the depth sensor in the first mode of operation by executing the first sequence of operation steps to generate the first depth measurements; and receive a second request for second depth measurements according to the second depth sensing mode of operation and, in response to the second request and without storing additional operation steps in the memory, operate the depth sensor in the second mode of operation by executing the second sequence of operation steps and the one or more common operation steps to generate the second depth measurements. ### Claim 2 (depends on 1) The depth sensor of claim 1, wherein the depth sensor is a time-of-flight camera. ### Claim 3 (depends on 1) The depth sensor of claim 1, wherein the first depth sensing mode of operation corresponds to a first distance range of measurements and the second depth sensing mode of operation corresponds to a second distance range of measurements which is different from the first distance range of measurements. ### Claim 4 (depends on 1) The depth sensor of claim 1, wherein the first depth measurements and the second depth measurements are usable in a virtual, augmented, or mixed reality display system. ### Claim 5 (depends on 1) The depth sensor of claim 1, wherein the first depth sensing mode of operation corresponds to a first frame rate and the second depth sensing mode of operation corresponds to a second frame rate which is different than the first frame rate. ### Claim 6 (depends on 1) The depth sensor of claim 1, wherein the first depth sensing mode of operation corresponds to a first frame rate and the second depth sensing mode of operation corresponds to a second frame rate which is slower than the first frame rate. ### Claim 7 (depends on 1) The depth sensor of claim 1, wherein at least one of the second sequence of operation steps comprises a delay. ### Claim 8 (depends on 1) The depth sensor of claim 1, wherein the processor is further configured to receive at least one configuration request and, in response to the at least one configuration request, store, in the memory, the first sequence of operation steps and the second sequence of operations steps. ### Claim 9 (depends on 1) The depth sensor of claim 1, wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a short-range and high frame rate mode of operation. ### Claim 10 (depends on 9) The depth sensor of claim 9, wherein the short-range and high frame rate mode of operation is used for sensing depths at ranges less than about 2 m with frame rates greater than about 20 Hz. ### Claim 11 (depends on 1) The depth sensor of claim 1, wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a short-range and low frame rate mode of operation. ### Claim 12 (depends on 11) The depth sensor of claim 11, wherein the short-range and low frame rate mode of operation is used for sensing depths at ranges less than about 2 m with frame rates less than about 20 Hz. ### Claim 13 (depends on 1) The depth sensor of claim 1, wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a long-range and high frame rate mode of operation. ### Claim 14 (depends on 13) The depth sensor of claim 13, wherein the long-range and high frame rate mode of operation is used for sensing depths at ranges from about 2-4 m with frame rates greater than about 20 Hz. ### Claim 15 (depends on 1) The depth sensor of claim 1, wherein the first depth sensing mode of operation or the second depth sensing mode of operation is a long-range and low frame rate mode of operation. ### Claim 16 (depends on 15) The depth sensor of claim 15, wherein the long-range and low frame rate mode of operation is used for sensing depths at ranges from about 2-4 m with frame rates less than about 20 Hz. ### Claim 17 (depends on 1) The depth sensor of claim 1, wherein: the first depth sensing mode of operation is a short-range and high frame rate mode of operation; the second depth sensing mode of operation is a short-range and low frame rate mode of operation; the one or more common operation steps include steps to: i) capture a short-range intensity sub-frame using a short exposure, and ii) capture four phase sub-frames; the first sequence of operation steps includes a first delay; and the second sequence of operation steps includes a second delay that is longer than the first delay. ### Claim 18 (depends on 1) The depth sensor of claim 1, wherein: the first depth sensing mode of operation is a long-range and high frame rate mode of operation; the second depth sensing mode of operation is a long-range and low frame rate mode of operation; the one or more common operation steps include steps to: i) capture a long-range intensity sub-frame using a long exposure, ii) capture four phase sub-frames for a first illumination modulation frequency, and iii) capture four phase sub-frames for a second illumination modulation frequency; the first sequence of operation steps includes a first delay; and the second sequence of operation steps includes a second delay that is longer than the first delay.