# US 11,113,513 B2 · Apparatus and method for training classification model and apparatus for classifying with classification model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,113,513 B2
- Document ID: US11113513B2
- Status: Granted
- Kind: B2
- Application number: 16/738,393
- Filed: 2020-01-09
- Published: 2020-08-20
- Granted: 2021-09-07
- Earliest claimed: 2019-02-19
- Assignees: Fujitsu Limited
- Applicants: Fujitsu Limited
- Inventors: Meng Zhang; Rujie Liu
- Web page: https://patentgrep.com/patent/US11113513B2

## CPC

- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/24: Classification techniques
- G06F 18/2413: based on distances to training or reference patterns
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06V 40/172: {Classification, e.g. identification}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An apparatus and method for training a classification model and an apparatus for classifying with a classification model are disclosed. The apparatus for training a classification model comprises: a local area obtainment unit to, obtain predetermined local area of each sample image; a feature extraction unit to, with respect to each sample image, set corresponding numbers of feature extraction layers for the global area and each predetermined local area, to extract a global feature of the global area and a local feature of each predetermined local area, wherein the global area and the predetermined local areas share at least one feature extraction layer in which the global feature and each local feature are combined; and a loss determination unit to calculate a loss function of the sample image based on combined features of each sample image, and to train the classification model based on the loss function.

## Claims

### Claim 1

An apparatus for training a classification model, comprising:
a local area obtainment unit configured to, with respect to each sample image in a training set, obtain a predetermined number of predetermined local areas of the sample image, wherein each predetermined local area is a part of a global area of the sample image;
a feature extraction unit configured to, with respect to each sample image, set corresponding numbers of feature extraction layers for the global area and each predetermined local area respectively, to extract a global feature of the global area and a local feature of each predetermined local area respectively, wherein the global area and the predetermined number of predetermined local areas share at least one layer of the corresponding numbers of feature extraction layers set with respect to the global area and each predetermined local area respectively, to combine the global feature and each local feature in the at least one shared layer; and
a loss determination unit configured to calculate, with a loss determination layer, a loss function of the sample image based on combined features of each sample image, and to train the classification model based on the loss function.

### Claim 2 (depends on 1)

The apparatus for training a classification model according to claim 1, wherein
the training set comprises a plurality of training sets, among which at least two training sets at least partially overlap,
the loss determination layer comprises a plurality of loss determination layers corresponding to the plurality of training sets respectively, and
the loss determination unit is configured to, with respect to each sample image from the plurality of training sets, calculate a loss function of the sample image with a loss determination layer corresponding to a training set to which the sample image belongs.

### Claim 3 (depends on 2)

The apparatus for training a classification model according to claim 2, wherein the plurality of training sets share at least one layer of feature extraction layers set with respect to the global area of the sample image in each training set.

### Claim 4 (depends on 3)

The apparatus for training a classification model according to claim 3, wherein the plurality of training sets also share at least one layer of feature extraction layers set with respect to at least one of the predetermined number of predetermined local areas of the sample image in each training set.

### Claim 5 (depends on 1)

The apparatus for training a classification model according to claim 1, wherein the predetermined number of local areas have complementary information about the global area.

### Claim 6 (depends on 1)

The apparatus for training a classification model according to claim 1, wherein the feature extraction layers are convolution layers of a convolutional neural network.

### Claim 7

A method for training a classification model, comprising:
a local area obtainment step of, with respect to each sample image in a training set, obtaining a predetermined number of predetermined local areas of the sample image, wherein each predetermined local area is a part of a global area of the sample image;
a feature extraction step of, with respect to each sample image, setting corresponding numbers of feature extraction layers for the global area and each predetermined local area respectively, to extract a global feature of the global area and a local feature of each predetermined local area respectively, wherein the global area and the predetermined number of predetermined local areas share at least one layer of the corresponding numbers of feature extraction layers set with respect to the global area and each predetermined local area respectively, to combine the global feature and each local feature in the at least one shared layer; and
a loss determination step of calculating, with a loss determination layer, a loss function of the sample image based on combined features of each sample image, and training the classification model based on the loss function.

### Claim 8 (depends on 7)

The method for training a classification model according to claim 7, wherein
the training set comprises a plurality of training sets, among which at least two training sets at least partially overlap,
the loss determination layer comprises a plurality of loss determination layers corresponding to the plurality of training sets respectively, and
in the loss determination step, with respect to each sample image from the plurality of training sets, a loss function of the sample image is calculated with a loss determination layer corresponding to a training set to which the sample image belongs.

### Claim 9 (depends on 8)

The method for training a classification model according to claim 8, wherein
the plurality of training sets share at least one layer of feature extraction layers set with respect to the global area of the sample image in each training set.

### Claim 10 (depends on 9)

The method for training a classification model according to claim 9, wherein the plurality of training sets also share at least one layer of feature extraction layers set with respect to at least one of the predetermined number of predetermined local areas of the sample image in each training set.

### Claim 11 (depends on 7)

The method for training a classification model according to claim 7, wherein the predetermined number of local areas have complementary information about the global area.

### Claim 12 (depends on 7)

The method for training a classification model according to claim 7, wherein the feature extraction layers are convolution layers of a convolutional neural network.

### Claim 13

An apparatus for classifying with a classification model, comprising:
a second local area obtainment unit configured to obtain a predetermined number of predetermined local areas of an image to be classified, wherein each predetermined local area is a part of a global area of the image to be classified;
a second feature extraction unit configured to extract, with corresponding numbers of feature extraction layers in the classification model which are set for the global area and each predetermined local area, a global feature of the global area and a local feature of each predetermined local area respectively, wherein in the classification model, the global area and the predetermined number of predetermined local areas share at least one layer of the corresponding numbers of feature extraction layers set with respect to the global area and each predetermined local area respectively, to combine the global feature and each local feature in the at least one shared layer; and
a second loss determination unit configured to calculate, with a loss determination layer in the classification model, a loss function of the image to be classified based on combined features of the image to be classified, and to classify the image to be classified based on the loss function.

### Claim 14 (depends on 13)

The apparatus for classifying with a classification model according to claim 13,
in a case where the classification model is obtained through training with a plurality of training sets, the loss determination layer in the classification model comprises a plurality of loss determination layers corresponding to the plurality of training sets respectively, and the plurality of training sets share at least one layer of feature extraction layers set with respect to the global area corresponding to each training set, and
the second loss determination unit is configured to respectively calculate, with each loss determination layer of the plurality of loss determination layers, a loss function of the image to be classified based on combined features of the image to be classified, and to classify the image to be classified based on the loss function.