# US 2022/0284543 A1 · Signal processing apparatus and signal processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284543 A1
- Document ID: US20220284543A1
- Status: Published application
- Kind: A1
- Application number: 17/597,528
- Filed: 2020-06-07
- Published: 2022-09-08
- Earliest claimed: 2019-07-19
- Assignees: Sony Semiconductor Solutions Corporation
- Applicants: Sony Semiconductor Solutions Corporation
- Inventors: KENSEI JO; TOMOHIRO MATSUKAWA; MITSUHARU OHKI; MASAHIRO WATANABE; KENICHI TAYU; KEITAROU AMAGAWA; KUMIKO MAHARA; TAKAHISA UENO
- Web page: https://patentgrep.com/patent/US20220284543A1

## CPC

- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- G01C 3/06: Use of electric means to obtain final indication
- G01S 17/89: for mapping or imaging
- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G06T 5/40: using histogram techniques
- G06V 30/18095: {Summing image-intensity values; Projection and histogram analysis}
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20216: Image averaging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A pixel is appropriately interpolated regarding a histogram image in which a distance histogram is allocated to each pixel position. A signal processing apparatus according to the present technology includes a histogram generation unit that inputs a histogram image in which a distance histogram indicating results of a plurality of times of distance measurement as frequency information for each distance is allocated to each pixel position and generates a distance histogram of an interpolation target position in the histogram image on the basis of distance histograms of a plurality of pixel positions near the interpolation target position.

## Claims

### Claim 1

A signal processing apparatus comprising:
a histogram generation unit configured to input a histogram image in which a distance histogram indicating results of a plurality of times of distance measurement as frequency information for each distance is allocated to each pixel position and generate a distance histogram of an interpolation target position in the histogram image on a basis of distance histograms of a plurality of pixel positions near the interpolation target position.

### Claim 2 (depends on 1)

The signal processing apparatus according to claim 1, wherein
the histogram generation unit
acquires information regarding a correlation direction that is a direction determined from a result of image correlation calculation performed for a plurality of directions in an image plane with reference to the interpolation target position and generates the distance histogram of the interpolation target position using weight information for each pixel position near the interpolation target position determined according to the correlation direction.

### Claim 3 (depends on 2)

The signal processing apparatus according to claim 2, wherein
the histogram generation unit
generates a histogram of the interpolation target position by averaging the distance histograms of the plurality of pixel positions near the interpolation target position by weighting based on the weight information for each pixel position.

### Claim 4 (depends on 1)

The signal processing apparatus according to claim 1, wherein
the histogram generation unit
generates the distance histogram of the interpolation target position using weight information determined according to a distance from the interpolation target position for each pixel position near the interpolation target position.

### Claim 5 (depends on 1)

The signal processing apparatus according to claim 1, wherein
the histogram generation unit
generates the distance histogram of the interpolation target position using weight information determined according to a luminance difference from the interpolation target position for each pixel position near the interpolation target position.

### Claim 6 (depends on 1)

The signal processing apparatus according to claim 1, wherein
the histogram generation unit
generates the distance histogram of the interpolation target position using weight information determined according to a distance from the interpolation target position for each pixel position near the interpolation target position and weight information determined according to a luminance difference from the interpolation target position for each pixel position near the interpolation target position.

### Claim 7 (depends on 1)

The signal processing apparatus according to claim 1, wherein
a histogram image input by the histogram generation unit is a reduced histogram image obtained by performing low-resolution conversion on an original histogram image, and
the histogram generation unit
inputs high resolution information that is information with a higher resolution than the reduced histogram image and is information in which information other than a distance histogram is allocated for each pixel position and generates a histogram image with a higher resolution than the reduced histogram image on a basis of the high resolution information and the reduced histogram image.

### Claim 8 (depends on 7)

The signal processing apparatus according to claim 7, wherein
the histogram generation unit
inputs the reduced histogram image generated by performing the low-resolution conversion performed by adding a distance histogram between adjacent pixels in the original histogram image.

### Claim 9 (depends on 7)

The signal processing apparatus according to claim 7, wherein
the high resolution information is
high resolution luminance information of which a distance histogram of each pixel position of the original histogram image is converted into a luminance.

### Claim 10 (depends on 9)

The signal processing apparatus according to claim 9, wherein
the high resolution luminance information is
information in which a value based on a sum of frequencies of respective distances of a distance histogram is allocated to each pixel position of the original histogram image.

### Claim 11 (depends on 7)

The signal processing apparatus according to claim 7, wherein
the high resolution information is
high-resolution direction information in which information regarding a correlation direction that is a direction determined for each pixel position from a result of image correlation calculation performed for a plurality of directions in an image plane for each pixel position of the original histogram image is allocated to each pixel position.

### Claim 12 (depends on 11)

The signal processing apparatus according to claim 11, wherein
the histogram generation unit
performs upsampling on the reduced histogram image by a bilinear interpolation method and executes image filter processing based on the correlation direction for each pixel position indicated by the high-resolution direction information on the upsampled histogram image.

### Claim 13 (depends on 1)

The signal processing apparatus according to claim 1, wherein
the histogram generation unit
inputs a histogram image generated on a basis of a reception signal from a light reception unit in which a plurality of pixels having a light reception element is two-dimensionally arranged as the histogram image and
generates a distance histogram of a pixel position of a defective pixel in the input histogram image on a basis of distance histograms of a plurality of pixel positions positioned near the pixel position of the defective pixel in the histogram image.

### Claim 14 (depends on 1)

The signal processing apparatus according to claim 1, wherein
the histogram generation unit
generates the distance histogram of the interpolation target position from the distance histograms of the plurality of pixel positions near the interpolation target position on a basis of a captured image obtained by imaging an object to be a distance measurement target of the input histogram image.

### Claim 15 (depends on 2)

The signal processing apparatus according to claim 2, wherein
the histogram generation unit
acquires information regarding the correlation direction determined from a result of the image correlation calculation with respect to a captured image obtained by imaging an object to be a distance measurement target of the input histogram image as information regarding the correlation direction.

### Claim 16

A signal processing method comprising:
inputting a histogram image in which a distance histogram indicating results of a plurality of times of distance measurement as frequency information for each distance is allocated to each pixel position and generating a distance histogram of an interpolation target position in the histogram image on a basis of distance histograms of a plurality of pixel positions near the interpolation target position.
