# US 2023/0055146 A1 · Methods for recognizing small targets based on deep learning networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0055146 A1
- Document ID: US20230055146A1
- Status: Published application
- Kind: A1
- Application number: 17/810,608
- Filed: 2022-07-03
- Published: 2023-02-23
- Earliest claimed: 2021-08-18
- Assignees: Zhejiang Gongshang University
- Applicants: Zhejiang Gongshang University
- Inventors: Huiyan WANG; Huan JIANG
- Web page: https://patentgrep.com/patent/US20230055146A1

## CPC

- G06T 7/0008: {checking presence/absence}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/82: using neural networks
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30168: Image quality inspection
- G06V 2201/06: Recognition of objects for industrial automation
- G06V 2201/07: Target detection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure provides a method for recognizing a small target based on a deep learning network. The method comprises: determining, based on a collected image, spot defect information through a recognition model including a first feature determination layer, a second feature determination layer, and a spot defect determination layer, determining, based on the collected image, a first feature map, determining, based on the first feature map, a second feature map by fusing with positional encoding, determining, based on the second feature map, a third feature map through the second feature determination layer, and obtaining, based on the third feature map, positional information of the spot defect through a first determination layer, and determining, based on the third feature map, classification information of the spot defect through a second determination layer.

## Claims

### Claim 1

A method for recognizing a spot defect, comprising:
determining spot defect information through a recognition model based on a collected image, the recognition model including a first feature determination layer, a second feature determination layer, and a spot defect determination layer, wherein
the second feature determination layer includes a self-attention layer used to determine an attention feature based on a Q matrix, a first K matrix, and a first V matrix, and the self-attention layer is further used to determine a first attention based on a QK distance and a Probability density;
the QK distance is a contraposition distance between a vector qij in the Q matrix and a vector kij in the K matrix; and
the probability density is determined through a probability density function based on the contraposition distance;
determining, based on the collected image, a first feature map through the first feature determination layer, and determining, based on the first feature map, a second feature map by fusing with positional encoding through the first feature determination layer;
determining, based on the second feature map, a third feature map through the second feature determination layer; and
determining, based on the third feature map, positional information of the spot defect through a first determination layer, and determining, based on the third feature map, classification information of the spot defect through a second determination layer.

### Claim 2 (depends on 1)

The method of claim 1, wherein
the Q matrix and the first K matrix are feature maps generated through a linear transformation of the second feature map, and
the first V matrix is generated through correlation calculation of the Q matrix and the first K matrix.

### Claim 3 (depends on 2)

The method of claim 2, wherein the self-attention layer is further used to:
generate a second K matrix through performing a dimension reduction of convolution on the first K matrix; and
generate a second V matrix through performing the dimension reduction of convolution on the first V matrix, wherein the second K matrix and the second V matrix are used for attention calculation.

### Claim 4 (depends on 2)

The method of claim 2, wherein the self-attention layer is further used to:
determine a second attention through calculation of matrix multiplication based on the Q matrix and a third K matrix; and
determine an attention matrix based on the first attention and the second attention, wherein the attention matrix is used for residual and normalization calculation and feedforward neural network calculation.

### Claim 5

(canceled)

### Claim 6 (depends on 1)

The method of claim 1, wherein the recognition model is obtained through training, and a loss function during the training includes a position loss item, the position loss item including an intersection-over-union of a label box and a prediction box and a position difference between the label box and the prediction box.

### Claim 7 (depends on 6)

The method of claim 6, wherein a weight ratio between the position difference and the intersection-over-union in the loss function is within a range of 2.4 to 2.6.

### Claim 8

A system for recognizing a spot defect, comprising
a non-transitory computer-readable storage medium storing at least one set of instructions; and
at least one processor in communication with the non-transitory computer-readable storage medium, when executing the at least one set of instructions, the at least one processor is directed to cause the system to:
determine a first feature map based on a collected image and determine a second feature map through fusing with positional encoding based on the first feature map;
determine a third feature map based on the second feature map, wherein the at least one processor is further directed to cause the system to:
determine an attention feature based on a Q matrix, a first K matrix, and a first V matrix, and determine a first attention based on a QK distance and a Probability density; wherein
the QK distance is a contraposition distance between a vector qij in the Q matrix and a vector kij in the K matrix; and
the probability density is determined through a probability density function based on the contraposition distance; and
determine positional information of the spot defect based on the third feature map and determine classification information of the spot defect based on the third feature map.

### Claim 9 (depends on 8)

The system of claim 8, wherein
the Q matrix and the first K matrix are feature maps generated through a linear transformation of the second feature map, and
the first V matrix is generated through correlation calculation of the Q matrix and the first K matrix.

### Claim 10 (depends on 9)

The system of claim 9, wherein the at least one processor is further directed to cause the system to:
generate a second K matrix through performing a dimension reduction of convolution on the first K matrix; and
generate a second V matrix through performing the dimension reduction of convolution on the first V matrix, wherein the second K matrix and the second V matrix are used for attention calculation.

### Claim 11 (depends on 9)

The system of claim 9, wherein the at least one processor is further directed to cause the system to:
determine a second attention through calculation of matrix multiplication based on the Q matrix and a third K matrix; and
determine an attention matrix based on the first attention and the second attention, wherein the attention matrix is used for residual and normalization calculation and feedforward neural network calculation.

### Claim 12

(canceled)

### Claim 13 (depends on 8)

The system of claim 8, wherein the at least one processor is further directed to cause the system to train a recognition model, wherein a loss function during the training includes a position loss item, and the position loss item includes an intersection-over-union of a label box and a prediction box and a position difference between the label box and the prediction box.

### Claim 14 (depends on 13)

The system of claim 13, wherein a weight ratio between the position difference and the intersection-over-union in the loss function is within a range of 2.4 to 2.6.

### Claim 15

A non-transitory computer readable storage medium, comprising at least one set of instructions, when executed by at least one processor of a computing device, causing the computing device to perform a method for recognizing a spot defect, the method comprising:
determining spot defect information through a recognition model based on a collected image, the recognition model including a first feature determination layer, a second feature determination layer, and a spot defect determination layer; wherein
the second feature determination layer includes a self-attention layer used to determine attention feature based on a Q matrix, a first K matrix, and a first V matrix, and the self-attention layer is further used to determine a first attention based on a QK distance and a Probability density;
the QK distance is a contraposition distance between a vector qij in the Q matrix and a vector kij in the K matrix; and
and the Probability density is determined through a Probability density function based on the contraposition distance;
determining, based on the collected image, a first feature map through the first feature determination layer, and determining, based on the first feature map, a second feature map by fusing with positional encoding through the first feature determination layer;
determining, based on the second feature map, a third feature map through the second feature determination layer;
determining, based on the third feature map, positional information of the spot defect through a first determination layer, and determining, based on the third feature map, classification information of the spot defect through a second determination layer.
