Back to results
US 11,637,978 B1Granted
Autonomous gating selection to reduce noise in direct time-of-flight depth sensing
AssigneeMeta Platforms Technologies, LLC
InventorsCharles Patrick Saunders, Michael Hall, Augusto Ronchini Ximenes
Life of this patent
Estimating the term…
Abstract
A depth camera assembly (DCA) includes a direct time of flight system for determining depth information for a local area. The DCA includes an illumination source, a camera, and a controller. In some embodiments, the controller uses previous image frames to determine confidence measurements, and selectively adjusts a number of pulses from the illuminator in a subsequent frame based on the determined confidence values. In some embodiments, the sensor uses autonomous gating, and the depth system includes a depth recovery pipeline which provide depth map estimates from the autonomous gated measurements.
Claims18
A depth camera assembly (DCA) comprising:
- an illumination source configured to illuminate a local area with outgoing light;
- an imaging device configured to capture image frames of the local area including reflected light comprising portions of the outgoing light reflected from the local area, the imaging device comprising:
- an array of pixels, and
- a plurality of counters coupled to a group of pixels of the array, each counter configured to increment in response to a pixel of the group detecting a photon, a first subset of the plurality of counters being incremented counters that are activated during capture of an image frame and a second subset of the plurality of counters being non-incremented counters that are deactivated during capture of the image frame;
- and
- a controller configured to, for the image frame:
- determine a histogram of a background photon count level for the image frame based on an acquisition time for each of the incremented counters;
- generate a convolution kernel matrix, each row of the convolution kernel matrix comprising a shift of a light pulse shape from the illumination source;
- augment the histogram by assigning a specific value to counters to non-incremented counters;
- and
- determine an arrival time of light from the illumination source to the imaging device based on a product of the convolutional kernel matrix and the augmented histogram.
8 more elements
A depth camera assembly (DCA) comprising:
- an illumination source transmitter configured to illuminate a local area with outgoing light;
- an imaging device configured to capture image frames of the local area including reflected light comprising portions of the outgoing light reflected from the local area, the imaging device comprising:
- an array of pixels, and
- a plurality of counters coupled to a group of pixels of the array, each counter configured to increment in response to a pixel of the group detecting a photon, a first subset of the plurality of counters being incremented counters that are activated during capture of an image frame and a second subset of the plurality of counters being non-incremented counters that are deactivated during capture of the image frame;
- and
- a controller configured to, for the image frame:
- determine a background photon count level for the image frame and a signal level for each of the incremented counters;
- generate a convolution kernel matrix, each row of the convolution kernel matrix comprising a shift of a light pulse shape from the illumination source plus the background photon count level;
- generate possible laser shifts by point-wise multiplying each row of the convolution kernel matrix by a binary vector having a value for each incremented counter and an alternate value for each non-incremented counter;
- determine a likelihood under a position distribution for each of the possible laser shifts;
- and
- determine a time of arrival of light from the illumination source to the imaging device from a possible laser shift having a largest likelihood.
9 more elements
A depth camera assembly (DCA), comprising:
- an illumination source transmitter configured to illuminate a local area with outgoing light;
- an imaging device configured to capture image frames of the local area including reflected light comprising portions of the outgoing light reflected from the local area, the imaging device comprising:
- a detector comprising an array of pixels, and
- a plurality of counters coupled to a group of pixels of the array, each counter configured to increment in response to a pixel of the group detecting a photon, a first subset of the plurality of counters being incremented counters that are activated during capture of an image frame and a second subset of the plurality of counters being non-incremented counters that are deactivated during capture of the image frame;
- and
- a controller configured to, for the image frame:
- determine a threshold confidence of light detected by the detector corresponding to light emitted by the illumination source;
- determine a confidence measure of light detected by a unit cell of the detector corresponding to light emitted by the illumination source for each unit cell of the detector;
- and
- modify emission of light from the illumination source to different regions of the local area corresponding to different unit cells of the detector based on comparison of the confidence measure determined for the unit cell of the detector to the threshold confidence.
7 more elements
Dependencies are inferred from the claim text.