# US 12,008,803 B2 · Neural network training method and apparatus for image retrieval, and electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,803 B2
- Document ID: US12008803B2
- Status: Granted
- Kind: B2
- Application number: 18/020,946
- Filed: 2020-08-25
- Published: 2023-09-21
- Granted: 2024-06-11
- Earliest claimed: 2020-08-13
- Assignees: Uisee Technologies (Zhejiang) Ltd.
- Applicants: Uisee Technologies (Zhejiang) Ltd.
- Inventors: Hao Ren; Siyang Li; Hong Lu
- Web page: https://patentgrep.com/patent/US12008803B2

## CPC

- G06N 3/045: Combinations of networks
- G06V 10/82: using neural networks
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/24317: {Piecewise classification, i.e. whereby each classification requires several discriminant rules}
- G06F 18/25: Fusion techniques
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A neural network training method and apparatus for image retrieval, and an electronic device are provided. A neural network includes: one feature extractor and a plurality of learners. The method includes: for each training image group, inputting three images of the training image group into the feature extractor, and determining features of the three images; for each image in each training image group, respectively multiplying the features of the image by a random weight corresponding to each learner, so as to obtain weighted features corresponding to each learner; for each image in each training image group, inputting the weighted features of the image corresponding to each learner into the corresponding learner, and determining a plurality of feature vectors of the image; and adjusting parameters of the neural network on based on the plurality of feature vectors of each image in a plurality of training image groups.

## Claims

### Claim 1

A neural network training method for an image retrieval, wherein a neural network comprises: one feature extractor and a plurality of learners, and the plurality of learners share features extracted by the one feature extractor; a plurality of training image groups are predetermined, and each training image group comprises three images, wherein two images are of the same category and one image is of a different category; and the method comprises:
for each training image group, assigning a random weight to each learner:
inputting the three images of the training image group into the feature extractor, and determining features of the three images;
for each image, respectively multiplying the features of the image by the random weight corresponding to each learner, so as to obtain weighted features of the image corresponding to each learner; and for each learner, making the three images in the training image group correspond to the same random weight;
for each image, inputting the weighted features of the image corresponding to each learner into the corresponding learner, and determining a plurality of feature vectors of the image; and
adjusting parameters of the neural network based on the plurality of feature vectors of each image in each training image group of the plurality of training image groups;
wherein a different random weight is reassigned to each learner every time when one training image group is changed to another training image group, while the random weight corresponding to each learner remains unchanged in processing the three images of the same training image group.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the plurality of training image groups are predetermined, which comprises:
acquiring a training sample set, wherein the training sample set comprises a plurality of categories of images, and each category comprises at least two images; and
determining a plurality of reference images A, an image P of the same category as each reference image A and an image N of a different category from each reference image A from the training sample set, so as to obtain the plurality of training image groups, wherein each training image group comprises: the reference image A, the image P of the same category and the image N of the different category.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the step of adjusting the parameters of the neural network based on the plurality of feature vectors of each image in each training image group of the plurality of training image groups, comprises:
for each image in each training image group, mapping the plurality of feature vectors of the image into an embedded feature vector through a fully connected layer;
for each training image group, calculating a loss function value of the training image group based on three embedded feature vectors corresponding to the three images and categories of the three images;
for the plurality of training image groups, determining an average loss function value of the plurality of training image groups; and
adjusting the parameters of the neural network based on the average loss function value.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the step of adjusting the parameters of the neural network based on the average loss function value, comprises:
adjusting the parameters of the neural network through a back propagation based on the average loss function value.

### Claim 5

A neural network training apparatus for image retrieval, wherein a neural network comprises: one feature extractor and a plurality of learners, and the plurality of learners share features extracted by the one feature extractor; the apparatus predetermines a plurality of training image groups, and each training image group comprises three images, wherein two images are of the same category and one image is of a different category; for each training image group, a random weight is assigned to each learner; and the apparatus comprises:
an extracting unit configured for, for each training image group, inputting the three images of the training image group into the feature extractor, and determining features of the three images;
a weighting unit configured for, for each training image group and for each image, respectively multiplying the features of the image by the random weight corresponding to each learner, so as to obtain weighted features of the image corresponding to each learner; and for each learner, making the three images in the training image group correspond to the same random weight;
a mapping unit configured for, for each training image group and for each image, inputting the weighted features of the image corresponding to each learner into the corresponding learner, and determining a plurality of feature vectors of the image; and
an adjusting unit configured for adjusting parameters of the neural network based on the plurality of feature vectors of each image in each training image group of the plurality of training image groups;
wherein a different random weight is reassigned to each learner every time when one training image group is changed to another training image group, while the random weight corresponding to each learner remains unchanged in processing the three images of the same training image group.

### Claim 6 (depends on 5)

The apparatus according to claim 5, wherein the apparatus predetermines the plurality of training image groups, which comprises that:
the apparatus acquires a training sample set, wherein the training sample set comprises a plurality of categories of images, and each category comprises at least two images; and
the apparatus determines a plurality of reference images A, an image P of the same category as each reference image A and an image N of a different category from each reference image A from the training sample set, so as to obtain the plurality of training image groups, wherein each training image group comprises: the reference image A, the image P of the same category and the image N of the different category.

### Claim 7 (depends on 5)

The apparatus according to claim 5, wherein the adjusting unit is configured for:
for each image in each training image group, mapping the plurality of feature vectors of the image into an embedded feature vector through a fully connected layer;
for each training image group, calculating a loss function value of the training image group based on three embedded feature vectors corresponding to the three images and categories of the three images;
for the plurality of training image groups, determining an average loss function value of the plurality of training image groups; and
adjusting the parameters of the neural network based on the average loss function value.

### Claim 8 (depends on 7)

The apparatus according to claim 7, wherein the adjusting unit adjusts the parameters of the neural network based on the average loss function value, which comprises:
adjusting the parameters of the neural network through a back propagation based on the average loss function value.

### Claim 9 (depends on 1)

An electronic device, comprising: a processor and a storage;
wherein the processor is configured for executing the steps of the method according to claim 1 by calling a program or an instruction stored in the storage.

### Claim 10 (depends on 1)

A non-transitory computer-readable storage medium, wherein the non-transient computer-readable storage medium stores a program or an instruction, and the program or the instruction enables a computer to execute the steps of the method according to claim 1.

### Claim 11 (depends on 9)

The electronic device according to claim 9, wherein in the method, the plurality of training image groups are predetermined, which comprises:
acquiring a training sample set, wherein the training sample set comprises a plurality of categories of images, and each category comprises at least two images; and
determining a plurality of reference images A, an image P of the same category as each reference image A and an image N of a different category from each reference image A from the training sample set, so as to obtain the plurality of training image groups, wherein each training image group comprises: the reference image A, the image P of the same category and the image N of the different category.

### Claim 12 (depends on 9)

The electronic device according to claim 9, wherein in the method, the step of adjusting the parameters of the neural network based on the plurality of feature vectors of each image in each training image group of the plurality of training image groups, comprises:
for each image in each training image group, mapping the plurality of feature vectors of the image into an embedded feature vector through a fully connected layer;
for each training image group, calculating a loss function value of the training image group based on three embedded feature vectors corresponding to the three images and categories of the three images;
for the plurality of training image groups, determining an average loss function value of the plurality of training image groups; and
adjusting the parameters of the neural network based on the average loss function value.

### Claim 13 (depends on 12)

The electronic device according to claim 12, wherein in the method, the step of adjusting the parameters of the neural network based on the average loss function value, comprises:
adjusting the parameters of the neural network through a back propagation based on the average loss function value.

### Claim 14 (depends on 10)

The non-transitory computer-readable storage medium according to claim 10, wherein in the method, the plurality of training image groups are predetermined, which comprises:
acquiring a training sample set, wherein the training sample set comprises a plurality of categories of images, and each category comprises at least two images; and
determining a plurality of reference images A, an image P of the same category as each reference image A and an image N of a different category from each reference image A from the training sample set, so as to obtain the plurality of training image groups, wherein each training image group comprises: the reference image A, the image P of the same category and the image N of the different category.

### Claim 15 (depends on 10)

The non-transitory computer-readable storage medium according to claim 10, wherein in the method, the step of adjusting the parameters of the neural network based on the plurality of feature vectors of each image in each training image group of the plurality of training image groups, comprises:
for each image in each training image group, mapping the plurality of feature vectors of the image into an embedded feature vector through a fully connected layer;
for each training image group, calculating a loss function value of the training image group based on three embedded feature vectors corresponding to the three images and categories of the three images;
for the plurality of training image groups, determining an average loss function value of the plurality of training image groups; and
adjusting the parameters of the neural network based on the average loss function value.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium according to claim 15, wherein in the method, the step of adjusting the parameters of the neural network based on the average loss function value, comprises:
adjusting the parameters of the neural network through a back propagation based on the average loss function value.