US 12,096,121 B2Granted

Precision reflectivity and ambient light removal for a geiger mode/single photon active sensor system

Assignee4D Intellectual Properties, LLC

InventorJames E. Retterath

Life of this patent

Estimating the term…

Loading patent…

Abstract

Geiger mode photo diodes are solid state photodetectors that are able to detect single photons. Such Geiger mode photo diodes are also referred to as single-photon detectors (SPDs). An array of SPDs can be used as a single detector element in an active sensing system, but sensor systems based on SPD arrays have at least two shortcomings due to ambient light. First, solar background light can hamper the ability to accurately determine depth. Second, ambient light impacts the reflectivity precision because of challenges differentiating between reflected light and ambient light. Embodiments enable active sensors that remove the ambient signal from a sensor's optical input. Other embodiments produce sensor intensity values that have higher precision than typical SPD array devices. Further embodiments produce sensor depth values that have higher precision than typical SPD array devices.

Claims18

Dependent claims

An active sensor system configured to generate a lighting-invariant depth map of a scene comprising:

at least one emitter configured to emit a set of light pulses toward the scene;
an array of detectors configured to receive light for a field of view that includes at least a portion of the scene, wherein each detector is configured to operate as a single-photon detector (SPD);
control circuitry operably coupled to the at least one emitter and the array of SPDs and configured to emit a set of light pulses and to capture a set of intensity values for at least three distance range bands and store the set of captured intensity values in a set of frame buffers;
and
a processing system operably coupled to the control circuitry and the set of frame buffers for the at least three distance range bands, the processing system configured to:
analyze the set of frame buffers for the at least three distance range bands to determine a minimum intensity value due to ambient light, a maximum intensity value, and a frame amongst the set of frame buffers for the at least three distance range bands at which the maximum intensity value occurs;
determine a depth based on the frame at which the maximum intensity value occurs;
determine a reflectivity value based on a difference between the maximum intensity value and the minimum intensity value;
and
generate a lighting invariant depth map of the scene based on the depth and the reflectivity value.

7 more elements

An active sensor system configured to generate a lighting-invariant image of a scene comprising:

at least one emitter configured to emit a set of active light pulses toward the scene;
an array of detectors configured to receive light for a field of view that includes at least a portion of the scene, wherein each detector is configured to operate as a single-photon detector (SPD);
control circuitry operably coupled to the at least one emitter and the array of SPDs and configured to store at least two successive frames of data as frame pixels in one or more frame buffers, wherein each frame pixel is based on a response from the array of SPDs, wherein a duration of a sampling period of the array of SPDs is constant for the at least two successive frames, wherein an intensity of the set of active light pulses emitted for each sampling period is different for the at least two successive frames;
and
a processing system operably coupled to the control circuitry and the one or more frame buffers to generate a lighting-invariant image of the scene, the processing system configured to:
analyze the at least two successive frames of data to determine a minimum intensity value due to ambient light in the scene and a maximum intensity value for the frame pixels;
and
generate a lighting-invariant image of the scene based on a difference between the maximum intensity value and the minimum intensity value for the frame pixels.

5 more elements

An active sensor system configured to generate a depth map of a scene comprising:

at least one emitter configured to emit a set of light pulses toward the scene;
an array of detectors configured to receive light for a field of view that includes at least a portion of the scene, wherein each detector is biased to operate as a Geiger mode detector;
control circuitry operably coupled to the at least one emitter and the array of detectors and configured to emit a set of light pulses and to capture a set of intensity values for at least three distance range bands and store the set of captured intensity values in a set of frame buffers;
and
a processing system operably coupled to the control circuitry and the set of frame buffers for the at least three distance range bands, the processing system configured to:
analyze the set of frame buffers and determine a minimum intensity value due to ambient light, a maximum intensity value, and a frame amongst the set of frame buffers at which the maximum intensity value occurs;
determine a depth based on the frame at which the maximum intensity value occurs;
determine a reflectivity value based on the difference between the maximum intensity value and the minimum intensity value;
and
generate a depth map of the scene based on the depth and the reflectivity value.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/417,282Filed 19 Jan 2024 · continuation
Priority: 3 Mar 2016
Published: US 2024/0147062 A123 May 2024
Granted: 17 Sep 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification12 codes

H04N 23/671
Focus control based on electronic image sensor signals: in combination with active ranging signals, e.g. using light or sound signals emitted…
G01S 7/4863
Circuits for detection, sampling, integration or read-out: detector arrays, e.g. charge-transfer gates
G01S 17/18
Using transmission of interrupted, pulse-modulated waves: wherein range gates are used
G01S 17/894
For mapping or imaging: three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver…

Family

Application15/059,811Parent of this continuation · filed 3 Mar 2016
Application15/853,222Parent of this continuation · filed 22 Dec 2017
US 9,866,816 B2Parent of this continuation · filed 9 Jan 2018
Application16/167,196Parent of this division · filed 22 Oct 2018
US 10,298,908 B2Parent of this continuation · filed 21 May 2019
Application16/537,305Parent of this continuation · filed 9 Aug 2019
US 10,382,742 B2Parent of this division · filed 13 Aug 2019
Application17/127,461Parent of this continuation · filed 18 Dec 2020
US 10,873,738 B2Parent of this continuation · filed 22 Dec 2020
Application17/967,365Parent of this continuation · filed 17 Oct 2022
US 11,477,363 B2Parent of this continuation · filed 18 Oct 2022
Application18/524,180Parent of this continuation · filed 30 Nov 2023
US 11,838,626 B2Parent of this continuation · filed 5 Dec 2023
US 2024/0171857 A1Pre-grant publication · published 23 May 2024
Application16/537,305Parent of this division
Application16/167,196Parent of this continuation
US 12,418,719 B2Related patent
US 2017/0257617 A1Published application
US 2018/0295344 A1Published application
US 2019/0058867 A1Published application
US 2020/0036958 A1Published application
US 2021/0258497 A1Published application
US 2023/0336869 A1Published application
US 2024/0147062 A1Published application
US 12,096,121 B2This patent

From the US filing history. Foreign family members aren’t listed.