# US 12,010,418 B2 · Solid-state imaging element, signal processing method thereof, and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,010,418 B2
- Document ID: US12010418B2
- Status: Granted
- Kind: B2
- Application number: 17/417,899
- Filed: 2019-12-25
- Published: 2022-03-24
- Granted: 2024-06-11
- Earliest claimed: 2019-01-08
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Tatsuo Fujiwara
- Web page: https://patentgrep.com/patent/US12010418B2

## CPC

- G06N 3/063: using electronic means
- H04N 23/61: based on recognised objects
- G06F 17/10: Complex mathematical operations {(function generation by table look-up G06F1/03; evaluation of elementary functions by calculation G06F7/544)}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- H04N 25/77: Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components
- H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present technology relates to a solid-state imaging element, a signal processing method thereof, and an electronic device capable of suppressing a bus band and power consumption required for reading a pixel signal from the solid-state imaging element. The solid-state imaging element performs calculation of at least one stage of a convolutional layer and a pooling layer in a convolutional neural network on a signal corresponding to a light reception amount of light incident on each pixel of a pixel array unit. The present technology can be applied to, for example, a solid-state imaging element and the like generating a captured image or a depth image.

## Claims

### Claim 1

A solid-state imaging element, comprising:
a calculation unit configured to:
execute a calculation, of at least one stage of a convolutional layer and a pooling layer in a convolutional neural network, on a signal corresponding to a light reception amount of light incident on each pixel of a pixel array unit;
change, based on a coefficient corresponding to the calculation of the convolutional layer, a capacity ratio of a capacitive element configured to hold the signal different for each pixel of the pixel array unit; and
multiply the signal by the coefficient corresponding to the calculation of the convolutional layer based on the changed capacity ratio of the capacitive element.

### Claim 2 (depends on 1)

The solid-state imaging element according to claim 1, further comprising:
a memory configured to store the coefficient corresponding to the calculation of the convolutional layer and the pooling layer, wherein
the calculation unit is further configured to execute the calculation of at least one stage of a convolutional layer and a pooling layer based on the multiplication of the signal of each pixel of the pixel array unit by the coefficient.

### Claim 3 (depends on 2)

The solid-state imaging element according to claim 2, wherein
the memory is further configured to store a plurality of types of the coefficient, and
the calculation unit is further configured to select one of the plurality of types of the coefficient for the calculation.

### Claim 4 (depends on 1)

The solid-state imaging element according to claim 1, further comprising:
an output unit configured to output, based on the calculation of at least one stage of the convolutional layer and the pooling layer, a pixel signal that has a resolution lower than a resolution corresponding to a number of pixels of the pixel array unit.

### Claim 5 (depends on 1)

The solid-state imaging element according to claim 1, further comprises the pixel array unit configured to capture an image, wherein
the calculation unit is further configured to:
extract a region of interest from the captured image, and
execute the calculation on the image in the region of interest.

### Claim 6

A signal processing method, comprising:
executing a calculation, by a solid-state imaging element, of at least one stage of a convolutional layer and a pooling layer in a convolutional neural network, on a signal corresponding to a light reception amount of light incident on each pixel of a pixel array unit;
changing, by the solid-state imaging element, based on a coefficient corresponding to the calculation of the convolutional layer, a capacity ratio of a capacitive element configured to hold the signal different for each pixel of the pixel array unit; and
multiplying, by the solid-state imaging element, the signal by the coefficient corresponding to the calculation of the convolutional layer based on the changed capacity ratio of the capacitive element; and
outputting, by the solid-state imaging element, a pixel signal based on the calculation.

### Claim 7

An electronic device, comprising:
a solid-state imaging element including:
a calculation unit configured to:
execute a calculation, of at least one stage of a convolutional layer and a pooling layer in a convolutional neural network, on a signal corresponding to a light reception amount of light incident on each pixel of a pixel array unit;
change, based on a coefficient corresponding to the calculation of the convolutional layer, a capacity ratio of a capacitive element configured to hold the signal different for each pixel of the pixel array unit; and
multiply the signal by the coefficient corresponding to the calculation of the convolutional layer based on the changed capacity ratio of the capacitive element.