# US 2025/0159333 A1 · Photodetection apparatus and electronic equipment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0159333 A1
- Document ID: US20250159333A1
- Status: Published application
- Kind: A1
- Application number: 18/835,039
- Filed: 2023-02-20
- Published: 2025-05-15
- Earliest claimed: 2022-03-08
- Assignees: Sony Semiconductor Solutions Corporation
- Applicants: Sony Semiconductor Solutions Corporation
- Inventors: Susumu HOGYOKU
- Web page: https://patentgrep.com/patent/US20250159333A1

## CPC

- G06T 1/0007: {Image acquisition}
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/063: using electronic means
- H04N 23/665: {involving internal camera communication with the image sensor, e.g. synchronising or multiplexing SSIS control signals}
- H04N 23/80: Camera processing pipelines; Components thereof
- H04N 25/443: by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming
- H04N 25/779: Circuitry for scanning or addressing the pixel array
- H04N 25/79: Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure relates to a photodetection apparatus and electronic equipment that are allowed to achieve increased efficiency. Bidirectional communication between an image sensor provided with a pixel group including N pixels from among a plurality of pixels arranged on a sensor surface in an array and a processor in which a neural network is formed by a neuron group with M neurons is performed by a group interface provided between the image sensor and the processor. The present technology can be applied to, for example, a stacked CMOS image sensor.

## Claims

### Claim 1

A photodetection apparatus comprising:
an image sensor provided with a pixel group including N (N is a positive integer) pixels from among a plurality of pixels arranged on a sensor surface in an array;
a processor in which a neural network is formed by a neuron group with M (M is a positive integer) neurons; and
a group interface provided between the pixel group and the neuron group and performing bidirectional communication between the image sensor and the processor.

### Claim 2 (depends on 1)

The photodetection apparatus according to claim 1, further comprising:
a pixel control section using a processing result of processing executed in the neural network to control properties of the pixel.

### Claim 3 (depends on 2)

The photodetection apparatus according to claim 2, wherein
the pixel control section adjusts any one or more of a gain, a delay, and a noise in a pixel signal output from the pixel and an unresponsive period of the pixel signal.

### Claim 4 (depends on 2)

The photodetection apparatus according to claim 2, wherein
the neural network executes processing of detecting a pixel signal with a fixed, temporally periodic pattern acquired by the image sensor.

### Claim 5 (depends on 2)

The photodetection apparatus according to claim 2, wherein
the pixel control section determines or changes a region-of-interest in which a subject of interest appears in an image captured by the image sensor, and the pixel control section sets the pixels other than those in the region-of-interest into a standby state.

### Claim 6 (depends on 2)

The photodetection apparatus according to claim 2, further comprising:
a firing detection section that, for each of the neurons in the neuron group, in a case where intermediate data determined by each of the neurons exceeds a predetermined threshold, detects firing of the neuron for which the intermediate data exceeds the threshold;
a learning section that determines a learning signal based on a firing rate of the neuron detected by the firing detection unit; and
a learning memory that accumulates the learning signal output from the learning section over time, wherein
a value obtained by accumulating the learning signal in the learning memory is fed to the pixel control section as a control signal and used to control the properties of the pixel.

### Claim 7 (depends on 6)

The photodetection apparatus according to claim 6, further comprising:
an operation section that performs a differential of the firing rate of the neuron or an operation corresponding to the differential and feeds a result of the operation to the firing detection section.

### Claim 8 (depends on 7)

The photodetection apparatus according to claim 7, wherein
the operation section performs the operation corresponding to the differential by use of at least any one of a leaky integrate value of the number of firings of the neuron and a membrane potential of the neuron.

### Claim 9 (depends on 7)

The photodetection apparatus according to claim 7, wherein
the operation section uses the operation corresponding to the differential to determine a time moving rate of the firing rate that is a differential, in a spatial direction, of a spatial distribution of firing rates of the neurons.

### Claim 10 (depends on 6)

The photodetection apparatus according to claim 6, wherein
the group interface accumulates the learning signal in the learning memory and performs, at a predetermined time, communication of transmitting the control signal.

### Claim 11 (depends on 1)

The photodetection apparatus according to claim 1, further comprising:
an address control circuit that executes address conversion processing of changing a virtual address of the corresponding neuron group to newly specifically associate the pixel group with the neuron group in response to receiving a content of a change in the pixel group transmitted from the image sensor in a case where the pixel group is changed, the content of the change being included in the communication.

### Claim 12 (depends on 11)

The photodetection apparatus according to claim 11, wherein,
when executing the address conversion processing, the address control circuit realizes the change in the pixel group by referencing a pre-held address table to replace the address.

### Claim 13 (depends on 1)

The photodetection apparatus according to claim 1, wherein,
at an interface between a semiconductor substrate on which the image sensor is provided and a semiconductor substrate on which the processor is provided, pieces of wiring are stacked by Cu—Cu bonding, and the group interface is arranged on the processor side of the interface.

### Claim 14 (depends on 13)

The photodetection apparatus according to claim 13, wherein
a neural network circuit different from the processor is connected to an outside of the photodetection apparatus, both the processor and the neural network circuit different from the processor communicate with each other, and both the image sensor and the neural network circuit different from the processor communicate with each other.

### Claim 15 (depends on 5)

The photodetection apparatus according to claim 5, wherein,
when the region-of-interest is output from the image sensor and input to the processor, the processor performs an operation of recognizing the subject appearing in the region-of-interest, and notifies the image sensor of the region-of-interest at a next time according to a result of the operation to change the region-of-interest to be output by the image sensor at the next time.

### Claim 16

Electronic equipment comprising:
an image sensor provided with a pixel group including N (N is a positive integer) pixels from among a plurality of pixels arranged on a sensor surface in an array;
a processor in which a neural network is formed by a neuron group with M (M is a positive integer) neurons; and
a group interface provided between the pixel group and the neuron group and performing bidirectional communication between the image sensor and the processor.
