# US 10,873,740 B2 · Three-dimensional image sensor, optical radar apparatus, and processing apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,740 B2 - Document ID: US10873740B2 - Status: Granted - Kind: B2 - Application number: 16/607,349 - Filed: 2018-04-06 - Published: 2020-04-30 - Granted: 2020-12-22 - Earliest claimed: 2017-04-28 - Assignees: Sharp Kabushiki Kaisha - Applicants: Sharp Kabushiki Kaisha - Inventors: Katsuji Iguchi - Web page: https://patentgrep.com/patent/US10873740B2 ## CPC - H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects - G01C 3/06: Use of electric means to obtain final indication - G01S 7/4863: Detector arrays, e.g. charge-transfer gates - G01S 7/4865: Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak (peak detection in noise, signal conditioning G01S7/487) - G01S 7/487: Extracting wanted echo signals {, e.g. pulse detection} - G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar - H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps - H10F 39/18: Complementary metal-oxide-semiconductor [CMOS] image sensors; Photodiode array image sensors - H10F 39/8057: {Optical shielding} - H10F 39/8063: {Microlenses} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - H10F: INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION ## Abstract An object is to realize a low-cost three-dimensional image sensor capable of providing a wide measurement range. The three-dimensional image sensor includes a light receiving unit including a plurality of pixels each including an SPAD (single-photon avalanche photodiode) configured to detect, in Geiger mode, a photon of incident light including reflected light from an object to be detected, a plurality of pixel storage elements provided for the respective pixels in a one-to-one manner, each pixel storage unit being configured to count a accumulated value of pulse signals output from a corresponding one of the pixels as a result of detecting photons for each of temporally successive time intervals, and a signal processing circuit (DS) configured to calculate distance information regarding a distance between a light receiving unit and an object for each pixel by using groups of accumulated values accumulated by the pixel storage elements. ## Claims ### Claim 1 An optical radar apparatus comprising: a light emitting element configured to emit pulsed light to illuminate an object to be detected, and a three-dimensional image sensor, wherein the three-dimensional image sensor comprises: a light receiving unit including a plurality of pixels each including an avalanche photodiode configured to detect, in Geiger mode, a photon of incident light including reflected light from an object to be detected; a plurality of accumulating units provided for the respective pixels in a one-to-one manner, each accumulating unit being configured to accumulate pulse signals output from a corresponding one of the pixels as a result of detecting photons for each of temporally successive time intervals; and a calculation unit configured to calculate distance information regarding a distance between the light receiving unit and the object for each pixel by using a time series of groups of accumulated values accumulated by the accumulating units, wherein the accumulating units are disposed outside the light receiving unit, wherein the avalanche photodiode in the three-dimensional image sensor detects a photon of reflected light of the pulsed light reflected from the object. ### Claim 2 (depends on 1) The optical radar apparatus according to claim 1, wherein the light emitting element emits pulsed light a plurality of times, and the accumulating unit of the three-dimensional image sensor accumulates a time series of groups of accumulated values acquired by the three-dimensional image sensor each time a plurality of pulses of light are emitted. ### Claim 3 (depends on 1) The optical radar apparatus according to claim 1, wherein the three-dimensional image sensor outputs distance information regarding the distance to the object, first light intensity information representing an intensity of reflected light of the pulsed light radiated to the object and reflected from the object, and second light intensity information that does not include reflected light of the pulsed light from the object. ### Claim 4 (depends on 3) A processing apparatus configured to recognize the object using the second light intensity information output from the optical radar apparatus according to claim 3. ### Claim 5 (depends on 3) A processing apparatus configured to detect the object approaching or moving away by detecting a time-dependent change in the first light intensity information output from the optical radar apparatus according to claim 3. ### Claim 6 (depends on 3) A processing apparatus configured to: using the distance information, the first light intensity information, and the second light intensity information, all output from the optical radar apparatus according to claim 3; in case where information from another image sensor is used together, recognize the object which is common in both the information from the other image sensor and the second light intensity information by comparing the information from the other image sensor and the second light intensity information; and when there is a large error which makes it difficult to determine the distance, detect a time-dependent change in the first light intensity information output. ### Claim 7 (depends on 1) The optical radar apparatus according to claim 1, wherein a length of the time interval is equal to or less than a full width at half maximum of the pulsed light. ### Claim 8 An optical radar apparatus comprising: a light emitting element configured to emit pulsed light to illuminate an object to be detected; and a three-dimensional image sensor, wherein the three-dimensional image sensor comprises: a light receiving unit including a plurality of pixels each including an avalanche photodiode configured to detect, in Geiger mode, a photon of incident light including reflected light from an object to be detected; a plurality of accumulating units provided for the respective pixels in a one-to-one manner, each accumulating unit being configured to accumulate pulse signals output from a corresponding one of the pixels as a result of detecting photons for each of temporally successive time intervals; and a calculation unit configured to calculate distance information regarding a distance between the light receiving unit and the object for each pixel by using a time series of groups of accumulated values accumulated by the accumulating units, wherein the accumulating units are disposed outside the light receiving unit, wherein the avalanche photodiode in the three-dimensional image sensor detects a photon of reflected light of the pulsed light reflected from the object, and the accumulating units each includes a plurality of binary counters for accumulating the pulse signals for each time interval. ### Claim 9 An optical radar apparatus comprising: a light emitting element configured to emit pulsed light to illuminate an object to be detected, and a three-dimensional image sensor, wherein the three-dimensional image sensor comprises: a light receiving unit including a plurality of pixels each including an avalanche photodiode configured to detect, in Geiger mode, a photon of incident light including reflected light from an object to be detected; a plurality of accumulating units provided for the respective pixels in a one-to-one manner, each accumulating unit being configured to accumulate pulse signals output from a corresponding one of the pixels as a result of detecting photons for each of temporally successive time intervals; and a calculation unit configured to calculate distance information regarding a distance between the light receiving unit and the object for each pixel by using a time series of groups of accumulated values accumulated by the accumulating units, wherein the accumulating units are disposed outside the light receiving unit, wherein the optical radar apparatus can output the distance information in middle of a frame in addition to the distance information at the end of the frame. ### Claim 10 (depends on 9) The optical radar apparatus according to claim 9, wherein the distance information in middle of a frame is about the object close to the optical radar apparatus. ### Claim 11 (depends on 9) The optical radar apparatus according to claim 9, wherein the distance information in middle of a frame is generated before overflow occur in the pixel storage elements of the three-dimensional image sensor.