# US 2020/0394416 A1 · Method and apparatus for training feature extraction model, computer device, and computer-readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394416 A1
- Document ID: US20200394416A1
- Status: Published application
- Kind: A1
- Application number: 17/007,507
- Filed: 2020-08-31
- Published: 2020-12-17
- Earliest claimed: 2018-07-27
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Guoping GONG; Xuyuan XU; Tao Wu
- Web page: https://patentgrep.com/patent/US20200394416A1

## CPC

- G06F 16/7837: {using objects detected or recognised in the video content}
- G06F 18/211: Selection of the most significant subset of features
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Aspects of the disclosure provide a method and an apparatus for training a feature extraction model, a computer device, and a computer-readable storage medium that belong to the field of video processing technologies. The method can include detecting a plurality of images in one or more sample videos and obtaining at least two images including the same object. The method can further include determining the at least two images including the same object as sample images, and training according to the determined sample images to obtain the feature extraction model, where the feature extraction model is used for extracting a video feature of the video.

## Claims

### Claim 1

A method for training a feature extraction model that is performed by a computer device, the method comprising:
detecting a plurality of images in one or more sample videos and obtaining at least two images including a same object, the at least two images indicating a change of the object in a time dimension;
determining the at least two images including the same object as sample images; and
training based on the determined sample images to obtain the feature extraction model that is configured to extracting a video feature of video.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the detecting the plurality of images in the one or more sample videos further comprises:
detecting a first image in the one or more sample videos to determine a first object included in the first image; and
tracking the first object in an image other than the first image to obtain a second image including the first object.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the tracking further comprises:
tracking the first object in the image other than the first image to obtain a plurality of other images including the first object; and
obtaining the second image including the first object from the plurality of other images by selecting either:
an image with a distance from the first image greater than a preset distance, where the distance between two images is a quantity of images between the two images, or a last image from the plurality of other images.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the training based on the determined sample images further comprises:
training based on the determined sample images and a preset loss function to obtain the feature extraction model, so that the feature extraction model meets a target that the preset loss function is minimum,
the preset loss function including at least the following information entropy loss function:
L
b
=
∑
d

(
u
-
0.5
)
2
;
,
where Lb represents an information entropy loss function, and u represents a sample feature average value of the determined sample images.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the preset loss function further comprises a distance loss function and a quantification loss function;
the distance loss function being:
Lt=max{(xa−xp)+m−(xa−xn),0}; and
the quantification loss function being:
L
q
=
{
x
2
,
x
<
0.5

(
x
-
1
)
2
,
x
≥
0.5
,
where Lt represents the distance loss function, xa represents a sample feature of any specified sample image, xp represents a sample feature of a sample image with a label the same as that of the specified sample image, xn represents a sample feature of a sample image with a label different from that of the specified sample image, m represents a distance constant, and m is greater than 0; Lq represents the distance loss function, x represents any value in a sample feature of any sample image, and x is greater than 0.

### Claim 6 (depends on 1)

The method according to claim 1, wherein after the training based on the determined sample images, the method further comprises:
obtaining at least one image in a target video; and
performing feature extraction on the at least one image based on the feature extraction model to obtain a video feature of the target video.

### Claim 7 (depends on 6)

The method according to claim 6, wherein after the performing the feature extraction on the at least one image based on the feature extraction model, the method further comprises:
storing the video feature and video information of the target video in a database; or
querying, in the database, the video information corresponding to the video feature, the database being configure to storing a video feature and video information of each video.

### Claim 8

An apparatus for training a feature extraction model, comprising processing circuitry that is configured to:
detect a plurality of images in one or more sample videos and obtain at least two images including the same object, the at least two images indicating a change of the object in a time dimension;
determine the at least two images including a same object as sample images; and
train based on the determined sample images to obtain the feature extraction model, the feature extraction model being configured to extract a video feature of video.

### Claim 9 (depends on 8)

The apparatus according to claim 8, wherein the processing circuitry is further configured to:
detect a first image in the sample video to determine a first object included in the first image; and
track the first object in an image other than the first image to obtain a second image including the first object.

### Claim 10 (depends on 9)

The apparatus according to claim 9, wherein the processing circuitry is further configured to:
track the first object in the image other than the first image to obtain a plurality of other images including the first object; and
obtain the second image including the first object from the plurality of other images by selecting either:
an image with a distance from the first image greater than a preset distance, where a distance between two images is a quantity of images between the two images, or a last image from the plurality of other images.

### Claim 11 (depends on 8)

The apparatus according to claim 8, wherein the processing circuitry is further configured to train according to the determined sample images and a preset loss function to obtain the feature extraction model so that the feature extraction model meets a target that the preset loss function is minimum,
the preset loss function including at least the following information entropy loss function:
L
b
=
∑
d

(
u
-
0.5
)
2
,
where Lb represents the information entropy loss function, and u represents a sample feature average value of the determined sample images.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein the preset loss function further comprises a distance loss function and a quantification loss function;
the distance loss function is as follows:
Lt=max{(xa−xp)+m−(xa−xn),0}; and
the quantification loss function is as follows:
L
q
=
{
x
2
,
x
<
0.5

(
x
-
1
)
2
,
x
≥
0.5
;
,
where Lt represents the distance loss function, xa represents a sample feature of any specified sample image, xp represents a sample feature of a sample image with a label the same as that of the specified sample image, xn represents a sample feature of a sample image with a label different from that of the specified sample image, m represents a distance constant, and m is greater than 0; Lq represents the distance loss function, x represents any value in a sample feature of any sample image, and x is greater than 0.

### Claim 13 (depends on 8)

The apparatus according to claim 8, wherein the processing circuitry is further configured to:
obtain at least one image in a target video; and
perform feature extraction on the at least one image based on the feature extraction model to obtain a video feature of the target video.

### Claim 14 (depends on 1)

A computer device, comprising a processor and a memory, the memory storing at least one instruction that, when executed by the processor, causes the processor to perform the method recited in claim 1.

### Claim 15 (depends on 14)

The computer device according to claim 14, wherein, when the operation of obtaining at least two images including the same object is performed, the computer device further performs:
detecting a first image in the sample video to determine a first object included in the first image; and
tracking the first object in an image other than the first image to obtain a second image including the first object.

### Claim 16 (depends on 14)

The computer device according to claim 14, wherein, when the operation of tracking the first object in the image other than the first image to obtain the second image including the first object is performed, the computer device further performs:
tracking the first object in the image other than the first image to obtain a plurality of other images including the first object; and
obtaining the second image including the first object from the plurality of other images by selecting either:
an image with a distance from the first image greater than a preset distance, where the distance between two images is a quantity of images between the two images, or a last image from the plurality of other images.

### Claim 17 (depends on 14)

The computer device according to claim 14, wherein, when the operation of training according to the determined sample images to obtain the feature extraction model is performed, the computer device further performs:
training according to the determined sample images and a preset loss function to obtain the feature extraction model, so that the feature extraction model meets a target that the preset loss function is minimum,
the preset loss function including at least the following information entropy loss function:
L
b
=
∑
d

(
u
-
0.5
)
2
,
where Lb represents the information entropy loss function, and u represents a sample feature average value of the determined sample images.

### Claim 18

A non-transitory computer-readable storage medium storing at least one instruction that, when executed by a processor, causes the processor to perform:
detecting a plurality of images in one or more sample videos and obtaining at least two images including a same object, the at least two images indicating a change of the object in a time dimension;
determining the at least two images including the same object as sample images; and
training based on the determined sample images to obtain an feature extraction model, the feature extraction model being used for extracting a video feature of video.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18, wherein, when the operation of obtaining at least two images including the same object is performed by the processor, the processor further performs:
detecting a first image in the sample video to determine a first object included in the first image; and
tracking the first object in an image other than the first image to obtain a second image including the first object.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium according to claim 19, wherein, when the operation of tracking the first object in the image other than the first image to obtain the second image including the first object is performed by the processor, the processor further performs:
tracking the first object in the image other than the first image to obtain a plurality of other images including the first object; and
obtaining the second image including the first object from the plurality of other images by selecting either:
an image with a distance from the first image greater than a preset distance, where a distance between two images is a quantity of images between the two images, or a last image from the plurality of other images.
