# US 12,010,434 B2 · Solid-state imaging apparatus and electronic device to perform arithmetic operation of neural network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,434 B2 - Document ID: US12010434B2 - Status: Granted - Kind: B2 - Application number: 17/608,764 - Filed: 2020-05-14 - Published: 2022-08-18 - Granted: 2024-06-11 - Earliest claimed: 2019-05-15 - Assignees: Sony Semiconductor Solutions Corporation - Applicants: Sony Semiconductor Solutions Corporation - Inventors: Hayato Wakabayashi; Ryoji Ikegaya - Web page: https://patentgrep.com/patent/US12010434B2 ## CPC - G06V 10/82: using neural networks - G06N 3/063: using electronic means - G06V 10/10: Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60) - G06V 10/12: Details of acquisition arrangements; Constructional details thereof - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V 10/955: {using specific electronic processors} - H04N 23/60: Control of cameras or camera modules - H04N 23/80: Camera processing pipelines; Components thereof - H04N 25/00: Circuitry of solid-state image sensors [SSIS]; Control thereof - H04N 25/571: involving a non-linear response - H04N 25/707: Pixels for event detection - H04N 25/77: Pixel circuitry, e.g. memories, A/D converters, pixel amplifiers, shared circuits or shared components - G06V 40/166: {using acquisition arrangements} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Provided is a solid-state imaging apparatus and an electronic device, in which a scale of circuits, to perform arithmetic operation of the neural network, is suppressed. A solid-state imaging apparatus according to an aspect of the present disclosure includes a pixel array unit and a processing unit. The pixel array unit has a plurality of first pixels that generate electric signals, which have a logarithmic characteristic with respect to light quantity, as first pixel signals. The processing unit performs arithmetic processing of the first neural network based on a plurality of first input data, which are based on the plurality of first pixel signals read from the pixel array unit, and a plurality of logarithmic weighting factors which express strength of the connection between the plurality of first nodes by a logarithm. ## Claims ### Claim 1 A solid-state imaging apparatus, comprising: a pixel array unit, wherein the pixel array unit includes: a plurality of first pixels configured to generate electric signals as a plurality of first pixel signals, and a plurality of second pixels different from the plurality of first pixels, and the generated electric signals have a logarithmic characteristic with respect to light quantity; a control unit configured to selectively execute one of a first read processing to read the plurality of first pixel signals from the plurality of first pixels or a second read processing to read a plurality of second pixel signals from the plurality of second pixels; and a processing unit configured to: perform, in a case of execution of the first read processing, arithmetic processing for a first neural network based on a plurality of pieces of first input data, and a plurality of logarithmic weighting factors which express strength of a connection between a plurality of first nodes of the first neural network by a logarithm, wherein the plurality of pieces of first input data is based on the plurality of first pixel signals read from the pixel array unit; and perform, in a case of execution of the second read processing, the arithmetic processing for a second neural network based on a plurality of pieces of second input data and a plurality of weighting factors that indicates strength of a connection between a plurality of second nodes of the second neural network, wherein the plurality of pieces of second input data is based on the plurality of second pixel signals. ### Claim 2 (depends on 1) The solid-state imaging apparatus according to claim 1, wherein each of the plurality of first pixel signals indicates a voltage value, the voltage value is in proportion to a logarithm of an electric current value to the base of a specific value, and the processing unit is configured to generate a piece of first input data of the plurality of pieces of first input data by multiplying the value of a first pixel signal of the plurality of first pixel signals by a logarithm of which an anti-logarithm is the specific value. ### Claim 3 (depends on 2) The solid-state imaging apparatus according to claim 2, wherein the specific value is a Napier's constant. ### Claim 4 (depends on 2) The solid-state imaging apparatus according to claim 2, wherein the base of the logarithm, of which the anti-logarithm is the specific value, is 2. ### Claim 5 (depends on 1) The solid-state imaging apparatus according to claim 1, wherein the control unit is further configured to select one of the first read processing or the second read processing, based on indication data that indicates one of the first read processing or the second read processing. ### Claim 6 (depends on 1) The solid-state imaging apparatus according to claim 1, wherein each of the plurality of second pixels is a pixel that is configured to generate an electric signal having a linear characteristic with respect to the light quantity, as a second pixel signal of the plurality of second pixel signals. ### Claim 7 (depends on 1) The solid-state imaging apparatus according to claim 1, wherein each of the plurality of second pixels is a pixel that is configured to generate a count value of a number of entered photons, as a second pixel signal of the plurality of second pixel signals. ### Claim 8 (depends on 1) The solid-state imaging apparatus according to claim 1, wherein each of the plurality of second pixels is a pixel that is configured to generate a charge amount, which indicates a phase difference of a reflected light of an emitted light from the emitted light, as a second pixel signal of the plurality of second pixel signals. ### Claim 9 (depends on 1) The solid-state imaging apparatus according to claim 1, wherein each pixel of the plurality of first pixels-pixel includes: a photoelectric conversion unit; a first read unit configured to read an electric signal having a logarithmic characteristic from the photoelectric conversion unit, as a first pixel signal of the plurality of first pixel signals; and a second read unit configured to read an electric signal having a linear characteristic from the photoelectric conversion unit, as a second pixel signal of the plurality of second pixel signals, and the control unit is further configured to selectively execute the first read processing to read the plurality of first pixel signals using the first read unit, or a second read processing to read the plurality of second pixel signals using the second read unit. ### Claim 10 (depends on 1) The solid-state imaging apparatus according to claim 1, wherein the processing unit is further configured to perform image recognition for images indicated by the plurality of first pixel signals, based on the arithmetic processing of the first neural network. ### Claim 11 An electronic device, comprising: an image pickup element that includes a pixel array unit, wherein the pixel array unit includes: a plurality of first pixels configured to generate electric signals as a plurality of first pixel signals, and a plurality of second pixels different from the plurality of first pixels, and the generated electric signals have a logarithmic characteristic with respect to light quantity; a control unit configured to selectively execute one of a first read processing to read the plurality of first pixel signals from the plurality of first pixels or a second read processing to read a plurality of second pixel signals from the plurality of second pixels; and a processing circuit configured to: perform, in a case of execution of the first read processing, arithmetic processing for a first neural network based on a plurality of pieces of first input data, and a plurality of logarithmic weighting factors which express strength of a connection between a plurality of first nodes of the first neural network by a logarithm, wherein the plurality of pieces of first input data is based on the plurality of first pixel signals read from the pixel array unit; and perform, in a case of execution of the second read processing, the arithmetic processing for a second neural network based on a plurality of pieces of second input data and a plurality of weighting factors that indicates strength of a connection between a plurality of second nodes of the second neural network, wherein the plurality of pieces of second input data is based on the plurality of second pixel signals.