# US 2022/0284703 A1 · Method and system for automated target recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284703 A1
- Document ID: US20220284703A1
- Status: Published application
- Kind: A1
- Application number: 17/687,251
- Filed: 2022-03-04
- Published: 2022-09-08
- Earliest claimed: 2021-03-05
- Assignees: DRS Network & Imaging Systems, LLC
- Applicants: DRS Network & Imaging Systems, LLC
- Inventors: Marco Lopez
- Web page: https://patentgrep.com/patent/US20220284703A1

## CPC

- G06V 10/82: using neural networks
- G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V 10/75: Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method includes receiving, from an image sensor, an image, identifying, by a first neural network, a plurality of locations-of-interest within the image, and generating, by the first neural network, a first classification label for each location-of-interest of the plurality of locations-of-interest. The method also includes extracting, from the image, a plurality of image chips derived from the plurality of locations-of-interest and generating, by a second neural network, a second classification label for each image chip of the plurality of image chips. The method further includes determining an identification of a set of targets within the image using the plurality of locations-of-interest, the first classification label for each location-of-interest of the plurality of locations-of-interest, the plurality of image chips, and the second classification label for each image chip of the plurality of image chips, and transmitting the identification of the set of targets within the image.

## Claims

### Claim 1

A method comprising:
receiving, from an image sensor, an image;
identifying, by a first neural network, a plurality of locations-of-interest within the image;
generating, by the first neural network, a first classification label for each location-of-interest of the plurality of locations-of-interest;
extracting, from the image, a plurality of image chips derived from the plurality of locations-of-interest;
generating, by a second neural network, a second classification label for each image chip of the plurality of image chips;
determining an identification of a set of targets within the image using the plurality of locations-of-interest, the first classification label for each location-of-interest of the plurality of locations-of-interest, the plurality of image chips, and the second classification label for each image chip of the plurality of image chips; and
transmitting the identification of the set of targets within the image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first neural network is a regions-based convolutional neural network (R-CNN).

### Claim 3 (depends on 1)

The method of claim 1, wherein the second neural network is a self-organizing map.

### Claim 4 (depends on 1)

The method of claim 1, wherein the second neural network is trained using a first partially-unsupervised training phase and a second supervised training phase.

### Claim 5 (depends on 1)

The method of claim 1, wherein the second neural network is configured to generate a label for each of a set of training images, and wherein the first neural network is trained using the set of training images and corresponding label.

### Claim 6 (depends on 1)

The method of claim 1, further comprising generating, for each image chip of the plurality of image chips, an image vector.

### Claim 7 (depends on 1)

The method of claim 1, further comprising generating, for each image chip of the plurality of image chips, a two-dimensional map representative of the image chip, before executing the second neural network.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the identification of the set of targets within the image includes:
determining, for a first subset of the first classification labels, that a confidence value associated with each of the first subset of the first classification labels, is greater than a first threshold;
determining, for a second subset of the second classification labels, that a score associated with each of the second subset of the second classification labels, is greater than a second threshold; and
defining the set of targets to include:
targets represented at locations-of-interest corresponding to the first subset of the first classification labels; and
targets represented in image chips corresponding to the second subset of the second classification labels.

### Claim 9 (depends on 1)

The method of claim 1, wherein the set of targets includes at least one overlapping potential target.

### Claim 10

A method comprising:
receiving a set of training images;
defining a first subset of training images of the set of training images, wherein training images in the first subset of training images are not labeled;
generating, from each training image in the first subset of training images, one or more image chips representing a portion of each training image and defining a region-of-interest;
aggregating, from each training image in the first subset of training images, the one or more image chips into a set of image chips;
for each image chip of the set of image chips:
converting the image chip into an image vector; and
generating, by executing a first neural network using the image vector, a classification label that corresponds to the image chip; and
training a second neural network using the set of image chips and classification labels, wherein the second neural network, once trained, is configured to identify one or more targets within a new image.

### Claim 11 (depends on 10)

The method of claim 10, wherein the first neural network is a self-organizing map.

### Claim 12 (depends on 10)

The method of claim 10, wherein the second neural network is a regions-based convolutional neural network (R-CNN).

### Claim 13 (depends on 10)

The method of claim 10, wherein generating the classification label that corresponds to the image chip includes:
determining a neuron of the first neural network that is closest to the image vector;
identifying a label that corresponds to the neuron; and
generating the classification label for the image vector, wherein the classification label corresponds to the label that corresponds to the neuron.

### Claim 14 (depends on 10)

The method of claim 10, wherein the first neural network is trained using semi-supervised learning.

### Claim 15 (depends on 10)

The method of claim 10, further comprising training the first neural network by:
defining a second subset of the set of training images, each image in the second subset of the set of training images being unlabeled;
executing an unsupervised learning phase using the set of training images, wherein the first neural network defines a temporary classification label for each training image in the second subset of the set of training images;
defining a third subset of the set of training images, wherein each training image in the third subset of the set of training images is associated with a new classification label that was not defined by the first neural network; and
executing a supervised learning phase using the third subset of the set of training images and the new classification labels.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
executing the first neural network using the second subset of the set of training images, wherein the first neural network generates an updated classification label for each training image in the second subset of the set of training images; and
replacing, for each training image in the second subset of the set of training images, the temporary classification label with the updated classification label.

### Claim 17 (depends on 15)

The method of claim 15, further comprising manually labeling each training image in the third subset of the set of training images.

### Claim 18 (depends on 10)

The method of claim 10, further comprising:
executing the second neural network using a particular image, wherein the second neural network generates an identification of one or more locations-of-interest and an identification of one or more potential targets within the particular image;
defining a new set of image chips from the one or more locations-of-interest;
executing the first neural network using the new set of image chips, wherein the first neural network generates, for each image chip of the new set of image chips, a particular classification label for the image chip; and
outputting, based on the identification of one or more potential targets and the particular classification label for each image chip of the one or more image chips, an identification of a set of targets within the particular image.

### Claim 19 (depends on 10)

The method of claim 10, wherein the region-of-interest is associated with a probability that the region-of-interest includes a representation of a target.
