# US 2025/0148760 A1 · Image sequence detection method and apparatus, medium, device, and program product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148760 A1
- Document ID: US20250148760A1
- Status: Published application
- Kind: A1
- Application number: 19/014,083
- Filed: 2025-01-08
- Published: 2025-05-08
- Earliest claimed: 2023-01-17
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Haoran QIAN
- Web page: https://patentgrep.com/patent/US20250148760A1

## CPC

- G06V 10/764: using classification, e.g. of video objects
- G06F 18/00: Pattern recognition
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/092: Reinforcement learning
- G06V 10/80: Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level (multimodal speaker identification or verification G10L17/10)
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V 2201/07: Target detection
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

This application discloses an image sequence detection method performed by an electronic device. The method includes: obtaining an initial image sequence, the initial image sequence including a target object; inputting the initial image sequence to a pretrained target detection model, to obtain a detection result sequence, the target detection model being obtained by inputting sample training data to an initial detection model to be trained for training, the sample training data including one set of sample feature information respectively extracted from one set of sample images, and each piece of sample feature information including feature information jointly determined by using one frame of sample image and a previous frame of sample image of the one frame of sample image; and determining a detection result corresponding to each image in the initial image sequence based on the detection result sequence, the detection result indicating a category of the target object.

## Claims

### Claim 1

An image sequence detection method performed by a computer device, the method comprising:
obtaining an initial image sequence, the initial image sequence comprising a target object;
inputting the initial image sequence to a pretrained target detection model, to obtain a detection result sequence; and
determining a detection result corresponding to each image in the initial image sequence based on the detection result sequence, the detection result indicating a category of the target object.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the target detection model is obtained by inputting sample training data to an initial detection model to be trained for training, the sample training data comprising one set of sample feature information respectively extracted from one set of sample images, and a classification result and an evaluation parameter that correspond to each piece of sample feature information, the classification result indicating a classification category for one piece of sample feature information, the evaluation parameter indicating a reward corresponding to the classification category, the one set of sample images being one set of continuous frames of images, and for an ith frame of sample image in the one set of continuous frames of images, sample feature information of the ith frame of sample image being jointly determined based on the ith frame of sample image and an (i−1)th frame of sample image.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising:
performing a first feature extraction operation on the one set of sample images, to obtain a first set of feature information, feature information in the first set of feature information being in one-to-one correspondence to sample images in the one set of sample images;
performing a second feature extraction operation on the one set of sample images, to obtain a second set of feature information, first feature information in the second set of feature information being in one-to-one correspondence to the sample images in the one set of sample images, and the first feature information of the ith frame of sample image indicating a correlation feature between the ith frame of sample image and the (i−1)th frame of sample image; and
fusing the first set of feature information and the second set of feature information respectively, to obtain the one set of sample feature information.

### Claim 4 (depends on 3)

The method according to claim 3, wherein for the ith frame of sample image, the performing a second feature extraction operation on the one set of sample images, to obtain a second set of feature information comprises:
performing an image recognition operation on the ith frame of sample image, to determine a first set of key points, the first set of key points being configured for describing a location of the target object in the ith frame of sample image;
performing an image recognition operation on the (i−1)th frame of sample image, to determine a second set of key points, the second set of key points being configured for describing a location of the target object in the (i−1)th frame of sample image; and
determining the first feature information of the ith frame of sample image based on the first set of key points and the second set of key points, the first feature information of the ith frame of sample image being configured for indicating a distance between corresponding key points in the first set of key points and the second set of key points.

### Claim 5 (depends on 2)

The method according to claim 2, wherein for the ith frame of sample image, the method further comprises:
inputting the sample feature information of the ith frame of sample image to the initial detection model, to obtain a classification result for the ith frame of sample image, the initial detection model being a detection model obtained by performing an initialization operation in advance; and
determining an evaluation parameter for the ith frame of sample image based on the classification result for the ith frame of sample image and a classification category corresponding to the sample feature information of the ith frame of sample image, the evaluation parameter for the ith frame of sample image being configured for indicating whether the classification result for the ith frame of sample image is the same as the corresponding classification category.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the inputting the sample feature information of the ith frame of sample image to the initial detection model, to obtain a classification result for the ith frame of sample image comprises:
inputting each category in a preset category set to a first preset function in sequence with the sample feature information of the ith frame of sample image, to obtain a first set of function values, the first set of function values comprising a function value for each category relative to the sample feature information of the ith frame of sample image; and
determining, as the classification result for the ith frame of sample image, a candidate category corresponding to a first function value in the first set of function values that is the largest.

### Claim 7 (depends on 6)

The method according to claim 6, wherein before the inputting each category in a preset category set to a first preset function in sequence with the sample feature information of the ith frame of sample image, to obtain a first set of function values, the method further comprises:
calculating a target random number in a preset manner;
determining whether the target random number satisfies a preset probability condition; and
selecting a candidate category randomly from the preset category set as the classification result for the ith frame of sample image when the target random number satisfies the preset probability condition; or
performing the operation of inputting each category in a preset category set to a first preset function in sequence with the sample feature information of the ith frame of sample image, to obtain a first set of function values, when the target random number does not satisfy the preset probability condition.

### Claim 8 (depends on 6)

The method according to claim 6, wherein the determining, as the classification result for the ith frame of sample image, a candidate category corresponding to a first function value in the first set of function values that is the largest comprises:
determining the evaluation parameter for the ith frame of sample image as a first evaluation parameter when the candidate category corresponding to the first function value is the same as the classification category for the ith frame of sample image, the first evaluation parameter indicating correct detection of the initial detection model; or
determining the evaluation parameter for the ith frame of sample image as a second evaluation parameter when the candidate category corresponding to the first function value is different from the classification category for the ith frame of sample image, the second evaluation parameter indicating incorrect detection of the initial detection model.

### Claim 9 (depends on 2)

The method according to claim 2, wherein the target detection model is obtained by:
selecting a plurality of sample data structures from the sample training data, a jth sample data structure in the plurality of sample data structures comprising a jth piece of sample feature information, a jth classification result corresponding to the jth piece of sample feature information, a jth evaluation parameter, and a (j+1)th piece of sample feature information, and j being a positive integer; and
training the initial detection model to be trained based on the plurality of sample data structures, to obtain the target detection model, the initial detection model being determined as the target detection model when a round count of training the initial detection model reaches a preset round count threshold, or a model parameter of the initial detection model being adjusted based on a predetermined loss function when a round count of training the initial detection model does not reach the preset round count threshold, and an input for a training process of each round being one of the plurality of sample data structures.

### Claim 10 (depends on 9)

The method according to claim 9, wherein for the jth sample data structure, the training the initial detection model to be trained based on the plurality of sample data structures, to obtain the target detection model comprises:
determining whether a sample image corresponding to the (j+1)th piece of sample feature information is a last sample image in the sample image sequence;
determining, based on the jth evaluation parameter when the sample image corresponding to the (j+1)th piece of sample feature information is the last sample image, a preset parameter outputted by the initial detection model; or
determining the preset parameter based on the jth evaluation parameter and an output of a reference detection model when the sample image corresponding to the (j+1)th piece of sample feature information is not the last sample image, the reference detection model being a detection model obtained by performing an initialization operation in advance, and a parameter of the reference detection model being different from a parameter of the initial detection model;
determining a value of the loss function based on the preset parameter and an output of the initial detection model, and updating the parameter of the initial detection model based on the value of the loss function; and
determining the initial detection model as the target detection model when a round count of performing the above operations reaches the preset round count threshold.

### Claim 11

An electronic device, comprising a memory and a processor, the memory having computer programs stored therein, and the computer programs, when executed by the processor, causing the electronic device to perform an image sequence detection method including:
obtaining an initial image sequence, the initial image sequence comprising a target object;
inputting the initial image sequence to a pretrained target detection model, to obtain a detection result sequence; and
determining a detection result corresponding to each image in the initial image sequence based on the detection result sequence, the detection result indicating a category of the target object.

### Claim 12 (depends on 11)

The electronic device according to claim 11, wherein the target detection model is obtained by inputting sample training data to an initial detection model to be trained for training, the sample training data comprising one set of sample feature information respectively extracted from one set of sample images, and a classification result and an evaluation parameter that correspond to each piece of sample feature information, the classification result indicating a classification category for one piece of sample feature information, the evaluation parameter indicating a reward corresponding to the classification category, the one set of sample images being one set of continuous frames of images, and for an ith frame of sample image in the one set of continuous frames of images, sample feature information of the ith frame of sample image being jointly determined based on the ith frame of sample image and an (i−1)th frame of sample image.

### Claim 13 (depends on 12)

The electronic device according to claim 12, wherein the method further comprises:
performing a first feature extraction operation on the one set of sample images, to obtain a first set of feature information, feature information in the first set of feature information being in one-to-one correspondence to sample images in the one set of sample images;
performing a second feature extraction operation on the one set of sample images, to obtain a second set of feature information, first feature information in the second set of feature information being in one-to-one correspondence to the sample images in the one set of sample images, and the first feature information of the ith frame of sample image indicating a correlation feature between the ith frame of sample image and the (i−1)th frame of sample image; and
fusing the first set of feature information and the second set of feature information respectively, to obtain the one set of sample feature information.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein for the ith frame of sample image, the performing a second feature extraction operation on the one set of sample images, to obtain a second set of feature information comprises:
performing an image recognition operation on the ith frame of sample image, to determine a first set of key points, the first set of key points being configured for describing a location of the target object in the ith frame of sample image;
performing an image recognition operation on the (i−1)th frame of sample image, to determine a second set of key points, the second set of key points being configured for describing a location of the target object in the (i−1)th frame of sample image; and
determining the first feature information of the ith frame of sample image based on the first set of key points and the second set of key points, the first feature information of the ith frame of sample image being configured for indicating a distance between corresponding key points in the first set of key points and the second set of key points.

### Claim 15 (depends on 12)

The electronic device according to claim 12, wherein for the ith frame of sample image, the method further comprises:
inputting the sample feature information of the ith frame of sample image to the initial detection model, to obtain a classification result for the ith frame of sample image, the initial detection model being a detection model obtained by performing an initialization operation in advance; and
determining an evaluation parameter for the ith frame of sample image based on the classification result for the ith frame of sample image and a classification category corresponding to the sample feature information of the ith frame of sample image, the evaluation parameter for the ith frame of sample image being configured for indicating whether the classification result for the ith frame of sample image is the same as the corresponding classification category.

### Claim 16 (depends on 15)

The electronic device according to claim 15, wherein the inputting the sample feature information of the ith frame of sample image to the initial detection model, to obtain a classification result for the ith frame of sample image comprises:
inputting each category in a preset category set to a first preset function in sequence with the sample feature information of the ith frame of sample image, to obtain a first set of function values, the first set of function values comprising a function value for each category relative to the sample feature information of the ith frame of sample image; and
determining, as the classification result for the ith frame of sample image, a candidate category corresponding to a first function value in the first set of function values that is the largest.

### Claim 17 (depends on 12)

The electronic device according to claim 12, wherein the target detection model is obtained by:
selecting a plurality of sample data structures from the sample training data, a jth sample data structure in the plurality of sample data structures comprising a jth piece of sample feature information, a jth classification result corresponding to the jth piece of sample feature information, a jth evaluation parameter, and a (j+1)th piece of sample feature information, and j being a positive integer; and
training the initial detection model to be trained based on the plurality of sample data structures, to obtain the target detection model, the initial detection model being determined as the target detection model when a round count of training the initial detection model reaches a preset round count threshold, or a model parameter of the initial detection model being adjusted based on a predetermined loss function when a round count of training the initial detection model does not reach the preset round count threshold, and an input for a training process of each round being one of the plurality of sample data structures.

### Claim 18 (depends on 17)

The electronic device according to claim 17, wherein for the jth sample data structure, the training the initial detection model to be trained based on the plurality of sample data structures, to obtain the target detection model comprises:
determining whether a sample image corresponding to the (j+1)th piece of sample feature information is a last sample image in the sample image sequence;
determining, based on the jth evaluation parameter when the sample image corresponding to the (j+1)th piece of sample feature information is the last sample image, a preset parameter outputted by the initial detection model; or
determining the preset parameter based on the jth evaluation parameter and an output of a reference detection model when the sample image corresponding to the (j+1)th piece of sample feature information is not the last sample image, the reference detection model being a detection model obtained by performing an initialization operation in advance, and a parameter of the reference detection model being different from a parameter of the initial detection model;
determining a value of the loss function based on the preset parameter and an output of the initial detection model, and updating the parameter of the initial detection model based on the value of the loss function; and
determining the initial detection model as the target detection model when a round count of performing the above operations reaches the preset round count threshold.

### Claim 19

A non-transitory computer-readable storage medium, comprising computer programs therein, wherein the computer programs, when executed by a processor of an electronic device, cause the electronic device to perform an image sequence detection method including:
obtaining an initial image sequence, the initial image sequence comprising a target object;
inputting the initial image sequence to a pretrained target detection model, to obtain a detection result sequence; and
determining a detection result corresponding to each image in the initial image sequence based on the detection result sequence, the detection result indicating a category of the target object.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium according to claim 19, wherein the target detection model is obtained by inputting sample training data to an initial detection model to be trained for training, the sample training data comprising one set of sample feature information respectively extracted from one set of sample images, and a classification result and an evaluation parameter that correspond to each piece of sample feature information, the classification result indicating a classification category for one piece of sample feature information, the evaluation parameter indicating a reward corresponding to the classification category, the one set of sample images being one set of continuous frames of images, and for an ith frame of sample image in the one set of continuous frames of images, sample feature information of the ith frame of sample image being jointly determined based on the ith frame of sample image and an (i−1)th frame of sample image.
