# US 2024/0203106 A1 · Feature extraction model processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203106 A1
- Document ID: US20240203106A1
- Status: Published application
- Kind: A1
- Application number: 18/592,346
- Filed: 2024-02-29
- Published: 2024-06-20
- Earliest claimed: 2022-10-11
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Binjie ZHANG; Yixiao GE; Shupeng SU; Xuyuan XU; Yexin WANG; Ying SHAN
- Web page: https://patentgrep.com/patent/US20240203106A1

## CPC

- G06V 10/776: Validation; Performance evaluation
- G06F 18/00: Pattern recognition
- G06V 10/40: Extraction of image or video features
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- Y02T 10/40: Engine management systems
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02T: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION

## Abstract

In a feature extraction model processing method, a sample image and an inheritance parameter of the sample image are obtained. The inheritance parameter is based on feature distinctiveness of a first image feature of the sample image. The first image feature is extracted from the sample image. A second image feature extracted from the sample image is obtained. A classification result of a first classification that is based on the second image feature is obtained. A classification loss of the first classification is adjusted based on the inheritance parameter to obtain a model compatibility loss. A classification loss of a second classification is determined. Respective model parameters of a feature extraction model to be trained and a image classification model to be trained are updated based on the model compatibility loss and the classification loss of the second classification.

## Claims

### Claim 1

A feature extraction model processing method, comprising:
obtaining a sample image and an inheritance parameter of the sample image, the inheritance parameter being based on feature distinctiveness of a first image feature of the sample image, and the first image feature being extracted from the sample image by a trained feature extraction model;
obtaining, from a feature extraction model to be trained, a second image feature extracted from the sample image;
obtaining, from a trained image classification model, a classification result of a first classification that is based on the second image feature, the trained image classification model being trained with the trained feature extraction model;
determining a classification loss of the first classification according to the classification result of the first classification;
adjusting the classification loss of the first classification based on the inheritance parameter to obtain a model compatibility loss;
obtaining, from an image classification model to be trained, a second classification result of a second classification based on the second image feature;
determining a classification loss of the second classification according to the classification result of the second classification; and
updating, based on the model compatibility loss and the classification loss of the second classification, respective model parameters of the feature extraction model to be trained and the image classification model to be trained.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
obtaining, from the trained feature extraction model, the first image feature extracted from the sample image;
obtaining, from the trained image classification model, an image category classification result of an image classification performed on the first image feature; and
determining the inheritance parameter of the sample image according to the image category classification result.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the determining the inheritance parameter comprises:
determining a category cross entropy parameter based on the image category classification result;
normalizing the category cross entropy parameter to obtain a distinctiveness parameter, the distinctiveness parameter indicating the feature distinctiveness of the first image feature; and
determining the inheritance parameter of the sample image according to the distinctiveness parameter.

### Claim 4 (depends on 1)

The method according to claim 1, wherein
the inheritance parameter includes an inheritance weight that has a positive correlation with a metric value of the feature distinctiveness, and
the adjusting the classification loss of the first classification includes weighting the classification loss of the first classification according to the inheritance weight to obtain the model compatibility loss.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the obtaining the classification result of the first classification comprises:
determining the trained image classification model and a category label of the sample image;
determining, based on the category label, a historical classification model parameter of the trained image classification model for a category to which the sample image belongs; and
obtaining, via the trained image classification model, the classification result of the first classification according to the second image feature and the historical classification model parameter.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the determining the classification loss of the first classification comprises:
obtaining the classification loss of the first classification based on an angle interval between the second image feature and the historical classification model parameter in the classification result of the first classification.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the determining the classification loss of the first classification comprises:
determining the classification loss of the first classification based on a difference between the classification result of the first classification and a category label of the sample image.

### Claim 8 (depends on 1)

The method according to claim 1, wherein
the obtaining the second classification result includes performing the second classification based on the second image feature via the image classification model to be trained; and
the determining the classification loss of the second classification includes determining, via the image classification model to be trained, the classification loss of the second classification based on a difference between the classification result of the second classification and a category label carried by the sample image.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the updating comprises:
determining a loss in training according to a sum of the model compatibility loss and the classification loss of the second classification; and
updating, based on the loss in the training, the respective model parameters of the feature extraction model to be trained and the image classification model to be trained.

### Claim 10 (depends on 1)

The method according to claim 1, further comprising:
performing feature mapping on the first image feature via a feature evolution model to be trained to obtain a mapping feature of the first image feature;
obtaining, from the image classification model to be trained, a third classification result of a third classification based on the mapping feature determining a classification loss of the third classification according to the classification result of the third classification,
wherein the updating includes updating, based on the model compatibility loss, the classification loss of the second classification, and the classification loss of the third classification, respective model parameters of the feature extraction model to be trained, the image classification model to be trained, and the feature evolution model to be trained.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the updating, based on the model compatibility loss, the classification loss of the second classification, and the classification loss of the third classification, the respective model parameters of the feature extraction model to be trained, the image classification model to be trained, and the feature evolution model to be trained comprises:
obtaining a target loss according to the model compatibility loss, the classification loss of the second classification, and the classification loss of the third classification; and
updating, based on the target loss, the respective model parameters of the feature extraction model to be trained, the image classification model to be trained, and the feature evolution model to be trained.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the obtaining the target loss comprises:
obtaining the target loss according to a sum of the model compatibility loss, the classification loss of the second classification, and the classification loss of the third classification.

### Claim 13 (depends on 10)

The method according to claim 10, further comprising:
determining a to-be-queried-image feature library associated with the trained feature extraction model, the to-be-queried-image feature library including respective to-be-queried-image features of images to be queried, and the to-be-queried-image features being extracted by the trained feature extraction model from the images to be queried;
performing feature mapping on the to-be-queried-image features through a trained feature evolution model to obtain respective to-be-queried-image mapping features of the images to be queried; and
updating the to-be-queried-image feature library based on the to-be-queried-image mapping features to obtain a to-be-queried-image feature library associated with the trained feature extraction model.

### Claim 14

A feature extraction method, comprising:
obtaining an image to be processed; and
performing, via a feature extraction model, feature extraction on the image to be processed to obtain a to-be-processed-image feature of the image to be processed, wherein
the feature extraction model obtained by updating, based on a model compatibility loss and a classification loss of a second classification, respective model parameters of a feature extraction model to be trained and an image classification model to be trained,
the model compatibility loss is obtained by a first classification that is performed based on a second image feature via a trained image classification model and adjustment of a classification loss of the first classification based on an inheritance parameter, the trained image classification model being trained with a trained feature extraction model;
the second image feature is extracted from a sample image via the feature extraction model to be trained;
the inheritance parameter is based on feature distinctiveness of a first image feature of the sample image;
the first image feature is extracted from the sample image by the trained feature extraction model; and
the classification loss of the second classification is obtained by the second classification that is performed based on the second image feature via the image classification model to be trained.

### Claim 15 (depends on 14)

The method according to claim 14, further comprising:
determining a to-be-queried-image feature library;
matching a feature in the to-be-queried-image feature library to the to-be-processed-image feature to obtain a to-be-queried-image feature matching the to-be-processed-image feature; and
determining, according to an image associated with the to-be-queried-image features, an image query result for the image to be processed.

### Claim 16

A feature extraction model processing apparatus, comprising:
processing circuitry configured to:
obtain a sample image and an inheritance parameter of the sample image, the inheritance parameter being based on feature distinctiveness of a first image feature of the sample image, and the first image feature being extracted from the sample image by a trained feature extraction model;
obtain, from a feature extraction model to be trained, a second image feature extracted from the sample image;
obtain, from a trained image classification model, a classification result of a first classification that is based on the second image feature, the trained image classification model being trained with the trained feature extraction model;
determine a classification loss of the first classification according to the classification result of the first classification;
adjust the classification loss of the first classification based on the inheritance parameter to obtain a model compatibility loss;
obtain, from an image classification model to be trained, a second classification result of a second classification based on the second image feature;
determine a classification loss of the second classification according to the classification result of the second classification; and
update, based on the model compatibility loss and the classification loss of the second classification, respective model parameters of the feature extraction model to be trained and the image classification model to be trained.

### Claim 17 (depends on 16)

The feature extraction model processing apparatus according to claim 16, wherein the processing circuitry is configured to:
obtain, from the trained feature extraction model, the first image feature extracted from the sample image;
obtain, from the trained image classification model, an image category classification result of an image classification performed on the first image feature; and
determine the inheritance parameter of the sample image according to the image category classification result.

### Claim 18 (depends on 17)

The feature extraction model processing apparatus according to claim 17, wherein the processing circuitry is configured to:
determine a category cross entropy parameter based on the image category classification result;
normalize the category cross entropy parameter to obtain a distinctiveness parameter, the distinctiveness parameter indicating the feature distinctiveness of the first image feature; and
determine the inheritance parameter of the sample image according to the distinctiveness parameter.

### Claim 19 (depends on 16)

The feature extraction model processing apparatus according to claim 16, wherein
the inheritance parameter includes an inheritance weight that has a positive correlation with a metric value of the feature distinctiveness, and
the processing circuitry is configured to weight the classification loss of the first classification according to the inheritance weight to obtain the model compatibility loss.

### Claim 20 (depends on 16)

The feature extraction model processing apparatus according to claim 16, wherein the processing circuitry is configured to:
determine the trained image classification model and a category label of the sample image;
determine, based on the category label, a historical classification model parameter of the trained image classification model for a category to which the sample image belongs; and
obtain, via the trained image classification model, the classification result of the first classification according to the second image feature and the historical classification model parameter.
