# US 12,192,652 B2 · Method and apparatus for estimating a value in a table generated by a photosites matrix

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,192,652 B2
- Document ID: US12192652B2
- Status: Granted
- Kind: B2
- Application number: 18/482,117
- Filed: 2023-10-06
- Published: 2024-02-08
- Granted: 2025-01-07
- Earliest claimed: 2020-03-18
- Assignees: STMicroelectronics France
- Applicants: STMicroelectronics France
- Inventors: Valentin Rebiere; Antoine Drouot
- Web page: https://patentgrep.com/patent/US12192652B2

## CPC

- G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
- H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values}
- H04N 25/13: characterised by the spectral characteristics of the filter elements
- H04N 25/60: Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N 25/705: Pixels for depth measurement, e.g. RGBZ
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An embodiment method for estimating a missing or incorrect value in a table of values generated by a photosite matrix comprises a definition of a zone of the table comprising the value to be estimated and other values, referred to as neighboring values, and an estimation of the value to be estimated based on the primary neighboring values and the weight associated with these primary neighboring values, wherein a weight of each neighboring value, referred to as primary neighboring value, of the same colorimetric component as that of the missing or incorrect value to be estimated, is determined according to differences between neighboring values disposed on an axis and neighboring values disposed parallel with this axis and positioned in relation to this axis on the same side as the primary neighboring value for which the weight is determined.

## Claims

### Claim 1

A method for estimating a missing or incorrect value, the method comprising:
defining a kernel of a table of values comprising a value to be estimated and neighboring values, the table of values being generated by a photosite matrix, the kernel having a missing or incorrect value location surrounded by locations of values generated by photosites, the missing or incorrect value location being the value to be estimated;
determining a weight for each primary neighboring value of a same colorimetric component as that of the missing or incorrect value to be estimated;
calculating semi-gradients, based on differences between neighboring values of the table, each semi-gradient being associated with a given direction with respect to the location of the missing or incorrect value to be determined, the weight of each primary neighboring value being determined based on the semi-gradient associated with a direction defined between the location of each primary neighboring value and the location of the missing or incorrect value; and
estimating the value to be estimated based on the primary neighboring values and the weight associated with the primary neighboring values.

### Claim 2 (depends on 1)

The method of claim 1, wherein the photosite matrix is a Bayer matrix.

### Claim 3 (depends on 1)

The method of claim 1, wherein the kernel has a size of 5×5 value locations.

### Claim 4 (depends on 1)

The method of claim 1, wherein eight semi-gradients are calculated for eight given directions along four axes passing through the location of the missing or incorrect value.

### Claim 5 (depends on 4)

The method of claim 4, wherein the four axes are one horizontal axis, one vertical axis, and two diagonal axes.

### Claim 6 (depends on 1)

The method of claim 1, wherein estimating the value comprises a discrimination of the semi-gradients by a Gaussian function, the weights of the primary neighboring values being calculated based on the discriminated semi-gradients.

### Claim 7 (depends on 1)

The method of claim 1, wherein estimating the value comprises a discrimination of the semi-gradients by a linear approximation of a Gaussian function.

### Claim 8 (depends on 1)

The method of claim 1, wherein the kernel is centered on the missing or incorrect value.

### Claim 9 (depends on 1)

The method of claim 1, wherein each semi-gradient is obtained from a sum of differences between neighboring values located on an axis passing through the missing or incorrect value and orthogonal to an orientation of the semi-gradient and neighboring values located in locations extending parallel with this axis and delimiting the kernel in the direction of the semi-gradient.

### Claim 10

A computer program product stored on a storage device, the computer program product comprising instructions to be executed by a microprocessor, the instructions when executed causing the microprocessor to:
define a kernel of a table of values comprising a value to be estimated and neighboring values, the table of values being generated by a photosite matrix, the kernel having a missing or incorrect value location surrounded by locations of values generated by photosites, the missing or incorrect value location being the value to be estimated;
determine a weight for each primary neighboring value of a same colorimetric component as that of the missing or incorrect value to be estimated;
define, in the table, several axes passing through a location of the missing or incorrect value to be estimated, the weight of each primary neighboring value being determined according to differences between neighboring values disposed in the table on one of the defined axes and neighboring values disposed in the table parallel with the one of the defined axes and positioned in relation to the one of the defined axes on a same side as the primary neighboring value for which the weight is determined;
calculate semi-gradients, based on the differences, each semi-gradient being associated with a given direction with respect to the location of the missing or incorrect value to be determined, the weight of each primary neighboring value being determined based on the semi-gradient associated with a direction defined between the location of each primary neighboring value and the location of the missing or incorrect value; and
estimate the value to be estimated based on the primary neighboring values and the weight associated with the primary neighboring values.

### Claim 11 (depends on 10)

The computer program product of claim 10, wherein the kernel is centered on the missing or incorrect value to be estimated.

### Claim 12 (depends on 10)

The computer program product of claim 10, wherein the kernel has a size of 5×5 value locations.

### Claim 13 (depends on 10)

The computer program product of claim 10, wherein the instructions are further configured to calculate at least two main gradients along two orthogonal orientations in relation to one another, each main gradient being calculated based on differences between the neighboring values disposed on one side of an axis passing through the location of the missing or incorrect value to be estimated and the neighboring values disposed on an other side of the axis, the axis being orthogonal to the orientation of the each main gradient.

### Claim 14 (depends on 13)

The computer program product of claim 13, wherein the instructions are further configured to compare between two mutually orthogonal main gradients so as to detect a dominant colorimetric rupture line passing through the location of the missing value.

### Claim 15 (depends on 10)

The computer program product of claim 10, wherein the photosites of the photosite matrix are disposed according to a Bayer matrix.

### Claim 16

An apparatus comprising:
a photosite matrix comprising photosites; and
a data acquisition and processing unit comprising a microprocessor, the data acquisition and processing unit being configured to:
acquire a table of values proportional to an illumination of the photosites,
wherein the table comprises a missing value at a location of one depth sensor, and wherein the missing value and each neighboring value of the table are associated with a colorimetric component;
define a kernel of the table comprising the missing value and the neighboring values, the location of the missing value being surrounded by locations of neighboring values;
determine a weight for each primary neighboring value of a same colorimetric component as that of the missing value;
define, in the table, several axes passing through a location of the missing value, wherein the weight of each primary neighboring value is determined according to differences between neighboring values disposed in the table on one of the defined axes and neighboring values disposed in the table parallel with the one of the defined axes and positioned in relation to the one of the defined axes on a same side as the primary neighboring value for which the weight is determined;
calculate semi-gradients, based on the differences, each semi-gradient being associated with a given direction with respect to the location of the missing value, the weight of each primary neighboring value being determined based on the semi-gradient associated with a direction defined between the location of each primary neighboring value and the location of the missing value;
estimate the missing value based on the primary neighboring values and the weight associated with the primary neighboring values; and
correct the missing value.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the apparatus is a mobile telephone.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein the apparatus is a tablet.

### Claim 19 (depends on 16)

The apparatus of claim 16, wherein each photosite is associated with red, green, or blue.

### Claim 20 (depends on 16)

The apparatus of claim 16, wherein the data acquisition and processing unit comprises a non-programmable integrated circuit.