# US 2021/0232813 A1 · Person re-identification method combining reverse attention and multi-scale deep supervision

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0232813 A1
- Document ID: US20210232813A1
- Status: Published application
- Kind: A1
- Application number: 17/027,241
- Filed: 2020-09-21
- Published: 2021-07-29
- Earliest claimed: 2020-01-23
- Assignees: Tongji University
- Applicants: Tongji University
- Inventors: Deshuang HUANG; Di WU
- Web page: https://patentgrep.com/patent/US20210232813A1

## CPC

- G06N 3/08: Learning methods
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present invention relates to a person re-identification method combining reverse attention and multi-scale deep supervision, including: constructing a person re-identification training network; training the person re-identification training network by using a training data set, to obtain a learning network, and discarding a reverse attention branch and a multi-scale deep supervision branch of a feature extraction module in the learning network, to obtain a test network; testing the test network by using a test data set, and after the test succeeds, inputting an actual data set into the learning network, to learn an image feature of the actual data set, and then discarding the reverse attention branch and the multi-scale deep supervision branch of the feature extraction module in the learning network, to obtain an application network; and inputting an actual query image into the application network, to obtain an identification result corresponding to the actual query image.

## Claims

### Claim 1

A person re-identification method combining reverse attention and multi-scale deep supervision, comprising the following steps:
S1. constructing a person re-identification training network comprising a feature extraction module and an identification output module, wherein a basic network of the feature extraction module uses a convolutional neural network ResNet50, and comprises a global branch, a reverse attention branch, and a multi-scale deep supervision branch;
S2. obtaining a training data set and a test data set;
S3. training the person re-identification training network by using the training data set, to obtain a person re-identification learning network; and shielding a reverse attention branch and a multi-scale deep supervision branch of a feature extraction module in the person re-identification learning network, to obtain a person re-identification test network;
S4. testing the person re-identification test network by using the test data set, and after the test succeeds, performing step S5; otherwise, returning to step S3;
S5. obtaining an actual data set and an actual query image;
S6. inputting the actual data set into the person re-identification learning network, to learn an image feature of the actual data set; then shielding the reverse attention branch and the multi-scale deep supervision branch of the feature extraction module in the person re-identification learning network, to obtain a person re-identification application network; and
S7. inputting the actual query image into the person re-identification application network, to obtain an identification result corresponding to the actual query image.

### Claim 2 (depends on 1)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 1, wherein the global branch in step S1 is used to extract global information of an image, comprising an attention module unit, an average pooling layer, and batch normalization that are sequentially connected, the attention module unit is divided into a first stage, a second stage, a third stage, and a fourth stage that are used for extracting a feature map, the attention module unit and the average pooling layer are combined to form a first global branch, and the attention module unit, the average pooling layer, and the batch normalization are combined to form a second global branch;
the reverse attention branch is used to extract, from feature maps extracted from the first stage to the fourth stage, feature information ignored by the attention module unit; and
the multi-scale deep supervision branch is used to extract feature information in horizontal and vertical directions from the feature maps extracted from the second stage and the third stage.

### Claim 3 (depends on 2)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 2, wherein the first global branch uses a ranked triplet loss function, and the second global branch uses an identity loss function; and
both the reverse attention branch and the multi-scale deep supervision branch use an identity loss function.

### Claim 4 (depends on 2)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 2, wherein the reverse attention branch comprises a reverse attention module unit and an average pooling layer that are sequentially connected, and input of the reverse attention module unit is separately output from the first stage to the fourth stage.

### Claim 5 (depends on 4)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 4, wherein the attention module unit comprises a channel attention module and a spatial attention module, and the channel attention module comprises one average pooling layer and two linear layers, to generate weight values corresponding to different channels; and
the spatial attention module comprises two dimension reduction layers and two convolutional layers, to enhance the significance of a feature at different spatial positions.

### Claim 6 (depends on 5)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 5, wherein specific calculation formulas of the attention module unit are as follows:
ATT=σ(ATTC×ATTS);
ATTC=BN(linear1(linear2(MC)));
ATTS=BN(Reduction2(Conv2(Conv1(MC))));
MC=Avgpool(M);
MϵC*W*H,MCϵC*1*1,
wherein ATT is an attention module, ATTC indicates channel attention that is output, ATTS indicates spatial attention that is output, linear1 is a first linear layer, linear2 is a second linear layer, BN the batch normalization, Conv2 and Conv1 respectively indicate two convolutional layers, Reduction2 indicates a second dimension reduction layer, AvgPool indicates an average pooling operation, M is a feature map, MC is a feature map on which average pooling is performed, C*W*H is a dimension of a feature map that is input, and C*1*1 is a dimension of a feature map obtained after average pooling operation is performed.

### Claim 7 (depends on 6)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 6, wherein a specific calculation formula of the reverse attention module unit is as follows:
ATTR=1−σ(ATTC×ATTS),
wherein ATTR is a reverse attention module.

### Claim 8 (depends on 1)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 1, wherein the multi-scale deep supervision branch comprises four convolution kernels on a one-dimensional scale, and sizes of the four convolution kernels on a one-dimensional scale are respectively 1×3, 3×1, 1×5, and 5×1.

### Claim 9 (depends on 3)

The person re-identification method combining reverse attention and multi-scale deep supervision according to claim 3, wherein the identity loss function is specified as follows:
L
ID
=
∑
i
=
1
N
⁢
-
q
i
⁢
log
⁡
(
p
i
)
;
q
i
=
{
1
-
(
N
-
1
)
⁢
ɛ
N
if
⁢
⁢
i
=
y
ɛ
N
otherwise
;
⁢
ɛ
=
0.1
,
wherein LID is an identity loss, pi is a prediction approximation, qi is a smooth identity weight, y is a true identity of a sample, i is an identity predicted by a network, N represents a quantity of training samples, and ε is a constant.
