# US 2023/0274529 A1 · Apparatus for optical image recognition and classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274529 A1
- Document ID: US20230274529A1
- Status: Published application
- Kind: A1
- Application number: 18/023,400
- Filed: 2020-08-28
- Published: 2023-08-31
- Earliest claimed: 2020-08-28
- Assignees: Technische UniversitäT Wien
- Applicants: Technische Universität Wien
- Inventors: Thomas Müller
- Web page: https://patentgrep.com/patent/US20230274529A1

## CPC

- G06V 10/147: Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19)
- G06V 10/764: using classification, e.g. of video objects
- G06F 18/24133: {Distances to prototypes}
- G06T 7/10: Segmentation; Edge detection (motion-based segmentation G06T7/215)
- G06V 10/10: Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60)
- G06V 10/20: Image preprocessing
- G06V 10/451: {with interaction between the filter responses, e.g. cortical complex cells}
- G06T 2207/20084: Artificial neural networks [ANN]
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The invention relates to an apparatus for optical image recognition and classification, comprising an optical setup (10) to split the optical image into n = 1, 2, N local optical power values Pn, a detector array (1) with n = 1, 2,... N photoactive pixels (2, 2′, 2″) to pick up the local optical power values Pn, wherein each pixel (2, 2′, 2″) is segmented into m = 1, 2,... M subpixels (3), and each subpixel (3) comprises a semiconductor photodiode which, under optical illumination, delivers a photocurrent Imn depending on its photoresponsivity Rmn and the local optical power Pn received at the pixel (2, 2′, 2″), and wherein the photoresponsivity values Rmn are not identical, and the outputs of each subpixel (3) are connected to form M output lines (4), wherein each output line (4) sums the photodetector currents produced by the m-th subpixels (3) in all N pixels (2, 2′, 2″) and delivers a detector current of Formel (I), to be used for recognizing and classifying the optical image.

## Claims

### Claim 1

Apparatus for optical image recognition and classification, comprising
an optical setup (10) to split the optical image into n = 1, 2, ..., N local optical power values Pn,
a detector array (1) with n = 1, 2, ... N photoactive pixels (2, 2′, 2″) to pick up the local optical power values Pn, wherein
each pixel (2, 2′, 2″) is segmented into subpixels (3), and
each subpixel (3) comprises a semiconductor photodiode which, under optical illumination, delivers a photocurrent depending on its photoresponsivity value and the local optical power Pn received at the pixel (2, 2′, 2″), characterized in that
the photoresponsivity values of the subpixels (3) within a pixel (2, 2′, 2″) are not identical, and
the outputs of the subpixels (3) of each pixel (2, 2′, 2″) are connected to form a number of separate output lines (4), wherein each output line (4) sums the photodetector currents produced by different subpixels (3) within each pixel (2, 2′, 2″) and delivers a number of detector currents to be used for recognizing and classifying the optical image.

### Claim 2 (depends on 1)

Apparatus according to claim 1, characterized in that each pixel (2, 2′, 2″) is segmented into m = 1, 2, ..., M subpixels (3), and the outputs of the subpixels (3) of each pixel (2, 2′, 2″) are connected to form M output lines (4).

### Claim 3 (depends on 1, 2)

Apparatus according to claim 1 or 2, characterized in that the number of subpixels (3) within each pixel (2, 2′, 2″) is not identical.

### Claim 4 (depends on 1, 2)

Apparatus according to claim 1 or 2, characterized in that each pixel (2, 2′, 2″) comprises an identical number of M subpixels (3).

### Claim 5 (depends on 1, 2, 3, 4)

Apparatus according to any of claims 1 to 4, characterized in that the photoresponsivity values Rmn of the subpixels (3) depend on the photoactive areas of the subpixel (3), and the photoactive areas of each subpixel (3) within a pixel (2, 2′, 2″) are not identical.

### Claim 6 (depends on 1, 2, 3, 4)

Apparatus according to any of claims 1 to 4, characterized in that the photoresponsivity values Rmn of the subpixels (3) are adjustable during operation, preferably by electrical means.

### Claim 7 (depends on 1, 2, 3, 4, 5, 6)

Apparatus according to any of claims 1 to 6, characterized in that the subpixels (3) each comprise a semiconductor channel (5), for example of Si, GaAs, or other semiconducting materials, which is contacted by a source contact (6) and a drain contact (7), wherein the source contact (6) is grounded, and the drain contact (7) forms the output of the subpixel (3).

### Claim 8 (depends on 6)

Apparatus according to claim 6, characterized in that a lateral p-n junction is formed in the channel (5) and two gate electrodes (8, 8′) are provided, the first gate electrode (8) located near the source contact (6), and the second gate electrode (8′) located near the drain contact, such that the built-in field over the p-n junction can be adjusted by applying a voltage at the gate electrodes.

### Claim 9 (depends on 6, 7)

Apparatus according to claim 6 or 7, characterized in that the channel (5) comprises a layered two-dimensional semiconductor such as a transition metal dichalcogenide, for example layered tungsten diselenide (WSe2) or layered molybdenum disulfide (MoS2) with less than five atomic layers and a total thickness of less than 10 nm, preferably less than 5 nm.

### Claim 10 (depends on 1, 2, 3, 4, 5, 6, 7, 8)

Apparatus according to any of claims 1 to 8, characterized in that the apparatus operates as an image classifier and comprises an electronic classifier circuitry (11) which is adapted to classify the optical image, according to the detector currents Im, into a number of y1, y2, ..., yM predefined categories.

### Claim 11 (depends on 1, 2, 3, 4, 5, 6, 7, 8, 9)

Apparatus according to any of claims 1 to 9, characterized in that the apparatus operates as an autoencoder, in particular as a binary-hashing autoencoder, to encode/decode Pn into Pn′, wherein the detector array (1) operates as encoder, and an electronic decoder circuitry (12) with weights Wnm is provided to decode the M values of Im into the N values of Pn′, wherein an electronic comparator unit (17) is provided to adjust the values of Rmn and Wnm to minimize the error between Pn and Pn′.

### Claim 12 (depends on 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)

Apparatus according to any of claims 1 to 10, characterized in that the apparatus comprises an electronic control circuitry (13), in particular an electronic memory, which is adapted to individually provide the photoresponsivity Rmn of each subpixel (3), in particular by setting voltages Vmn and -Vmn at the first gate electrode (8) and the second gate electrode (8′).

### Claim 13 (depends on 11)

Apparatus according to claim 11, characterized in that the control circuitry is adapted to set the voltage at the first gate electrode (8) to +VG and at the second gate electrode (8′) to -VG, with VG at around 1 V to 5 V, resulting in a range of the photoresponsivity Rmn of preferably -1 A/W to + 1 A/W, such as -60 mA/W to +60 mA/W.

### Claim 14 (depends on 7, 8, 9, 10, 11, 12)

Apparatus according to any of claims 7 to 12, characterized in that the subpixels (3) each comprise two electrically isolated, floating gate electrodes (9, 9′), the first floating gate electrode (9) being located near the first gate electrode (8) and the second floating gate electrode (9′) being located near the second gate electrode (8′), wherein the isolation layer between the floating gate electrodes (9, 9′) and the gate electrodes (8, 8′) is sufficiently thin so that they can be charged when a voltage is applied to the gate electrodes (8, 8′).

### Claim 15 (depends on 13)

Apparatus according to claim 13, characterized in that the gate electrodes (8, 8′) are embedded in an isolating layer of Al2O3, and the floating gates (9, 9′) are embedded in an isolating layer of hexagonal boron nitride (hBN).

### Claim 16 (depends on 13, 14)

Apparatus according to claim 13 or 14, characterized in that the floating gate electrodes (9, 9′) are around 2-nm-thick layers of Au.

### Claim 17 (depends on 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)

Apparatus according to any of claims 1 to 16, characterized in that each pixel (2, 2′, 2″) is segmented into a linear array or a rectangular matrix of M subpixels (3).

### Claim 18 (depends on 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)

Artificial neural network for optical image recognition and classification, characterized in that it comprises an apparatus according to any of claims 1 to 17, wherein the subpixels (3) are used as artificial neurons and the photoresponsivity values Rmn of the subpixels (3) are used as weights of the artificial neural network.

### Claim 19 (depends on 18)

Artificial neural network according to claim 18, characterized in that the values Rmn of the subpixels (3) are initially set during fabrication of the detector array (1) by designing the photoactive areas of each subpixel (3) within a pixel (2, 2′, 2″) at a predetermined value, resulting in a predetermined value of Rmn for each subpixel (3).

### Claim 20 (depends on 19)

Artificial neural network according to claim 19, characterized in that the values Rmn of the subpixels (3) are adjustable during operation by electrical means, in particular by applying voltages at gate electrodes (8, 8′).

### Claim 21 (depends on 20)

Artificial neural network according to claim 20, characterized in that the values Rmn of the subpixels (3) are adjustable during a training period by electrical means, in particular by consecutively applying voltages at gate electrodes (8, 8′), whereby electric charge is stored at floating gate electrodes (9, 9′) which are assigned to the gate electrodes (8, 8′).
