# US 12,000,931 B2 · Continuous wave time of flight system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,000,931 B2 - Document ID: US12000931B2 - Status: Granted - Kind: B2 - Application number: 17/316,285 - Filed: 2021-05-10 - Published: 2021-11-18 - Granted: 2024-06-04 - Earliest claimed: 2020-05-15 - Assignees: Analog Devices International Unlimited Company - Applicants: Analog Devices International Unlimited Company - Inventors: Jonathan Ephraim David Hurwitz; Krystian Balicki; Eoin E. English - Web page: https://patentgrep.com/patent/US12000931B2 ## CPC - G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar - G01B 11/22: for measuring depth - G01S 7/4814: {of transmitters alone} - G01S 7/4816: {of receivers alone} - G01S 7/491: Details of non-pulse systems - G01S 7/4911: Transmitters - G01S 17/36: with phase comparison between the received signal and the contemporaneously transmitted signal - H04N 25/44: by partially reading an SSIS array - H04N 25/46: by combining or binning pixels - H04N 25/709: Circuitry for control of the power supply - H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems and methods for a lower energy/faster time of flight camera system configured to measure distance to an object. The system is configured to emit laser light modulated with a first frequency and image light reflected by an object in order to determine a first phase difference between the emitted and reflected light. The system emits laser light modulated with a second frequency and images light reflected by the object to determine a second phase difference between the emitted and reflected light. The distance to the object is determined using the first and second phase differences. The system is arranged to operate at lower energy for obtaining the first phase difference compared with the operation to obtain the second phase difference. This results in lower overall energy consumption and faster overall operation without any significant reduction in accuracy of imaging. ## Claims ### Claim 1 A continuous wave time of flight, CW-ToF, camera system comprising: a laser for outputting laser light; an imaging sensor comprising a plurality of imaging pixels for accumulating charge based on incident light comprising reflected laser light; and an image acquisition system coupled to the imaging sensor and configured to: acquire, while operating in a low energy mode, a first set of charge samples from the imaging sensor by being configured to: a) drive the laser to output laser light modulated with a first modulation signal, wherein the first modulation signal has a first frequency; and b) read out image sensor values indicative of charge accumulated by at least some of the plurality of imaging pixels of the imaging sensor; acquire, while operating in a high energy mode, a second set of charge samples from the imaging sensor by being configured to: c) drive the laser to output laser light modulated with a second modulation signal, wherein the second modulation signal has a second frequency, and wherein the second frequency is greater than the first frequency; and d) read out image sensor values indicative of charge accumulated by at least some of the plurality of imaging pixels of from the imaging sensor that is greater than the plurality of imaging pixels read out while operating in the lower energy mode; and determine a depth frame based on the first set of charge samples and the second set of charge samples. ### Claim 2 (depends on 1) The CW-ToF camera system of claim 1 wherein the image acquisition system is configured to: in step b), read out image sensor values from a first set of imaging pixels of the imaging sensor; in step d), read out image sensor values from a second set of imaging pixels of the imaging sensor; wherein the first set of imaging pixels comprises fewer imaging pixels than the second set of imaging pixels. ### Claim 3 (depends on 1) The CW-ToF camera system of claim 1, wherein in the low energy mode, to acquire the first set of charge samples, the image acquisition system is configured to: bin the image sensor values of groups of two or more imaging pixels, such that each of the first set of charge samples comprises a plurality of binned pixel values. ### Claim 4 (depends on 3) The CW-ToF camera system of claim 3, wherein in step b) the image acquisition system is configured to: bin, in the analog domain, image sensor values of groups of two or more imaging pixels; and digitally convert the plurality of binned pixel values. ### Claim 5 (depends on 2) The CW-ToF camera system of claim 2, wherein the image acquisition system is further configured to determine a depth frame based on the first set of charge samples and the second set of charge samples, and wherein to determine the depth frame comprises to upsample the first set of charge samples. ### Claim 6 (depends on 2) The CW-ToF camera system of claim 2, the image acquisition system is further configured to determine a depth frame based on the first set of charge samples and the second set of charge samples, including being configured to: use the first set of charge samples to determine phase differences between the output laser light modulated with a first modulation frequency and light incident on the imaging sensor; and upsample the determined phase differences between the output laser light modulated with a first modulation frequency and light incident on the imaging sensor. ### Claim 7 (depends on 1) The CW-ToF camera system of claim 1, wherein the low energy mode comprises driving the laser such that an optical power of the output laser light modulated with the first modulation signal is lower than an optical power of the output laser light modulated with the second modulation signal. ### Claim 8 (depends on 1) The CW-ToF camera system of claim 1, wherein the image acquisition system comprises one or more amplifiers for amplifying signals read from the imaging sensor, and wherein the image acquisition system is configured to: operate at least some of the one or more amplifiers in a first power mode when acquiring the first set of charge samples; and operate at least some of the one or more amplifiers in a second power mode when acquiring the second set of charge samples, wherein the first power mode is lower than the second power mode. ### Claim 9 (depends on 1) The CW-ToF camera system of claim 1, wherein the image acquisition system comprises one or more analog-to-digital converters, ADC, for converting analog signals read from the imaging sensor to digital signals, and wherein the image acquisition system is configured to: operate at least some of the ADCs in a first power mode when determining at least some of the first set of samples; and operate at least some of the ADCs in a second power mode when determining at least some of the second set of samples, wherein the first power mode is lower than the second power mode. ### Claim 10 (depends on 1) The CW-ToF camera system of claim 1, wherein CW-ToF is configured to: repeat steps a) and b) a first plurality of times such that the acquired first set of charge samples comprises a first plurality of charge samples; and repeat steps c) and d) a second plurality of times such that the acquired second set of charge samples comprises a second plurality of charge samples; wherein the first plurality is less than the second plurality, such that the first plurality of charge samples comprises fewer charge samples than the second plurality of charge samples. ### Claim 11 (depends on 1) The CW-ToF camera system of claim 1, wherein in the low energy mode the CW-ToF is configured to: drive the laser in steps a) and c) such that laser light is output from the laser during step a) for a shorter period of time than laser light is output from the laser during step c). ### Claim 12 (depends on 1) The CW-ToF camera system of claim 1, wherein the image acquisition system is further configured to determine a depth frame based on the first set of charge samples and the second set of charge samples. ### Claim 13 (depends on 1) The CW-ToF camera system of claim 1, wherein the image acquisition system is further configured to output the first set of charge samples and the second set of charge samples to a processor for determination of a depth frame. ### Claim 14 A method of obtaining a plurality of charge samples using a continuous wave time of flight ToF, CW-ToF, camera system comprising a laser and an imaging sensor having a plurality of imaging pixels for accumulating charge based on incident light, the method comprising: acquiring a first set of charge samples from the imaging sensor while operating in a low energy mode by: a) driving the laser to output laser light modulated with a first modulation signal, wherein the first modulation signal has a first frequency; and b) reading out image sensor values indicative of charge accumulated by at least some of the plurality of imaging pixels of the image sensor; acquiring a second set of charge samples from the imaging sensor while operating in a high energy mode by: c) driving the laser to output laser light modulated with a second modulation signal, wherein the second modulation signal has a second frequency, and wherein the second frequency is greater than the first frequency; and d) reading out image sensor values indicative of charge accumulated by at least some of the plurality of imaging pixels of from the image sensor that is greater than the plurality of imaging pixels read out while operating in the lower energy mode. ### Claim 15 (depends on 14) The method of claim 14, further comprising determining a depth image based on the first set of charge samples and the second set of charge samples. ### Claim 16 An image acquisition system for use in a continuous wave time of flight, CW-ToF, camera system, the image acquisition system being configured to: obtain, while operating in a low energy mode, a first plurality of charge samples by being configured to: drive a laser to emit first laser light modulated with a first modulation signal using a relatively low energy mode; and read out image sensor values from a first plurality of imaging pixels of an imaging sensor; obtain, while operating in a high energy mode, the second plurality of charge samples by being configured to: drive the laser to emit second laser light modulated with a second modulation signal using a second energy mode; and read out image sensor values from a second plurality of imaging pixels of the imaging sensor, wherein each of the first plurality of charge samples comprises fewer data points obtained during the low energy mode compared with each of the second plurality of charge samples obtained during the high energy mode. ### Claim 17 (depends on 16) The image acquisition system of claim 16, wherein the first plurality of imaging pixels is fewer in number than the second plurality of imaging pixels. ### Claim 18 (depends on 16) The image acquisition system of claim 16, wherein to obtain the first plurality of charge samples the image acquisition system is configured to bin the image sensor values of groups of two or more imaging pixels such that each of the first plurality of charge samples comprises a plurality of binned pixel values, wherein the plurality of binned pixel values in each of the first plurality of charge samples is fewer than a number of image sensor values in each of the second plurality of charge samples. ### Claim 19 (depends on 16) The image acquisition system of claim 16, wherein the image acquisition system is configured to obtain the first plurality of charge samples by being configured to: drive the laser to emit first laser light modulated with the first modulation signal for a plurality of periods of time; for each of the plurality of periods of time, control the imaging sensor to accumulate charge on at least some of the imaging pixels for a portion of the modulation signal, wherein the portion of the modulation signal for which accumulation takes place has a different phase offset for each of the plurality of periods of time; and at the conclusion of each of the plurality of periods of time, obtain one of the first plurality of charge samples by reading out image sensor values from the first plurality of imaging pixels. ### Claim 20 (depends on 19) The image acquisition system of claim 19, wherein the same set of imaging pixels of the imaging sensor are read out for each of the first plurality of charge samples.