# US 2023/0237790 A1 · Meta-optic accelerators for object classifiers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0237790 A1
- Document ID: US20230237790A1
- Status: Published application
- Kind: A1
- Application number: 18/159,832
- Filed: 2023-01-26
- Published: 2023-07-27
- Earliest claimed: 2022-01-26
- Assignees: Vanderbilt University
- Applicants: Vanderbilt University
- Inventors: Jason Valentine; Yuankai Huo; Hanyu Zheng; Quan Liu
- Web page: https://patentgrep.com/patent/US20230237790A1

## CPC

- G06V 10/82: using neural networks
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/895: {the filter being related to phase processing, e.g. phase-only filters}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system for identifying objects in images is provided. The system may include an optical front end and a digital back end. The optical front end includes a metalens that duplicates a received image into multiple images, and a metasurface that receives the duplicate images and outputs a feature map based on the received images. The feature map may be equivalent to the computationally expensive convolution operations previously performed by a neural network. The feature map is provided to the digital back end, which uses a neural network to classify the object. Because the feature map included the convolution operations, the digital back end can classify the object more quickly and using fewer computing resources than previous systems.

## Claims

### Claim 1

An object classification system comprising:
an optical front end adapted to;
receive an image of an object; and
output a feature map, or maps, based on the image; and
a digital back end adapted to:
receive the feature map from the optical front end;
process the feature map using a neural network; and
output a classification for the object.

### Claim 2 (depends on 1)

The system of claim 1, wherein the optical front end comprises:
a lens adapted to duplicate the received image into a plurality of images; and
a kernel layer adapted to receive the plurality of images and to output the feature map based on the plurality of images.

### Claim 3 (depends on 2)

The system of claim 2, wherein the plurality of images comprises multiple images.

### Claim 4 (depends on 2)

The system of claim 2, wherein the lens is a multi-channel metalens, and the kernel layer 117 is a metasurface.

### Claim 5 (depends on 2)

The system of claim 2, wherein the kernel layer replaces a multifunctional kernel layer of the neural network.

### Claim 6 (depends on 2)

The system of claim 2, wherein the metasurface encodes polarization and amplitude information for convolution with the plurality of images.

### Claim 7 (depends on 2)

The system of claim 2, further comprising:
receiving a set of training data; and
training the metasurface and the neural network using the set of training data.

### Claim 8 (depends on 1)

The system d of claim 1, wherein the neural network comprises a shallow or deep digital neural network.

### Claim 9 (depends on 1)

The system of claim 1, wherein the classification for the object comprises a histogram.

### Claim 10 (depends on 1)

The system of claim 1, wherein the system comprises a digital camera.

### Claim 11

A method for classifying an object in an image comprising:
receiving an image of an object by an optical front end;
outputting a feature map, or maps, based on the image by the optical front end;
receiving the feature map from the optical front end by a digital back end;
processing the feature map using a neural network by the digital back end; and
outputting a classification for the object by the digital back end.

### Claim 12 (depends on 11)

The method of claim 11, wherein outputting a feature map based on the image by the optical front end comprises:
duplicating the received image into a plurality of images by a lens; and
outputting the feature map based on the plurality of images by a metasurface.

### Claim 13 (depends on 12)

The method of claim 12, wherein the plurality of images comprises nine or more images.

### Claim 14 (depends on 12)

The method of claim 12, wherein the lens is a multi-channel metalens.

### Claim 15 (depends on 12)

The method of claim 12, wherein the metasurface replaces a multifunctional kernel layer of the neural network.

### Claim 16 (depends on 12)

The method of claim 12, wherein the metasurface encodes polarization and amplitude information for convolution with the plurality of images.

### Claim 17 (depends on 12)

The method of claim 12, further comprising:
receiving a set of training data; and
training the metasurface and the neural network using the set of training data.

### Claim 18 (depends on 11)

The method of claim 11, wherein the neural network comprises a shallow or deep digital neural network.

### Claim 19 (depends on 11)

The method of claim 11, wherein the classification for the object comprises a histogram.

### Claim 20

A digital camera comprising:
an optical front end adapted to;
receive an image of an object; and
output a feature map, or maps, based on the image; and
a digital back end adapted to:
receive the feature map from the optical front end;
process the feature map using a neural network; and
output a classification for the object, wherein the optical front end comprises a
lens adapted to duplicate the received image into a plurality of images; and
a metasurface adapted to receive the plurality of images and to output the feature map based on the plurality of images, and wherein the metasurface replaces a multifunctional kernel layer of the neural network.
