# US 2021/0342632 A1 · Image processing method and apparatus, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342632 A1
- Document ID: US20210342632A1
- Status: Published application
- Kind: A1
- Application number: 17/378,931
- Filed: 2021-07-19
- Published: 2021-11-04
- Earliest claimed: 2019-03-25
- Assignees: Shanghai Sensetime Intelligent Technology Co., Ltd.
- Applicants: Shanghai Sensetime Intelligent Technology Co., Ltd.
- Inventors: Jiafei WU; Mingliang LIANG
- Web page: https://patentgrep.com/patent/US20210342632A1

## CPC

- G06V 40/168: {Feature extraction; Face representation}
- G06F 18/2113: by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
- G06F 18/251: {of input or preprocessed data}
- G06F 18/253: {of extracted features}
- G06V 10/806: {of extracted features}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An image processing method includes: obtaining an image feature of each of a plurality of images for a same object; determining, according to the image feature of each of the plurality of images, a weight coefficient having a one-to-one correspondence with each image feature; and performing feature fusion processing on the image features of the plurality of images based on the weight coefficient of each image feature to obtain a fusion feature of the plurality of images.

## Claims

### Claim 1

An image processing method, comprising:
obtaining an image feature of each of a plurality of images for a same object;
determining, according to the image feature of each of the plurality of images, a weight coefficient having a one-to-one correspondence with each image feature; and
performing feature fusion processing on the image features of the plurality of images based on the weight coefficient of each image feature to obtain a fusion feature of the plurality of images.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining, according to the image feature of each of the plurality of images, the weight coefficient having the one-to-one correspondence with each image feature comprises:
forming an image feature matrix based on the image feature of each of the plurality of images;
performing feature fitting processing on the image feature matrix to obtain a first weight matrix; and
determining the weight coefficient corresponding to each image feature based on the first weight matrix.

### Claim 3 (depends on 2)

The method of claim 2, wherein performing the feature fitting processing on the image feature matrix to obtain the first weight matrix comprises:
performing the feature fitting processing on the image feature matrix using a regularized least-square linear regression algorithm, and obtaining the first weight matrix under condition that a preset target function takes a minimum value.

### Claim 4 (depends on 2)

The method of claim 2, wherein determining the weight coefficient corresponding to each image feature based on the first weight matrix comprises:
determining each of first weight coefficients comprised in the first weight matrix as the weight coefficient corresponding to each image feature; or performing first optimizing processing on the first weight matrix, and
determining each of first weight coefficients comprised in an optimized first weight matrix as the weight coefficient corresponding to each image feature.

### Claim 5 (depends on 4)

The method of claim 4, wherein performing the first optimizing processing on the first weight matrix comprises:
determining a fitting image feature of each of the plurality of images based on the first weight coefficient of each of the image features comprised in the first weight matrix, wherein the fitting image feature is a product of the image feature and the first weight coefficient corresponding to the image feature,
performing the first optimizing processing on the first weight matrix using a first error between the image feature of each of the plurality of images and the fitting image feature of the image to obtain a first optimized weight matrix;
in response to that a difference between the first weight matrix and the first optimized weight matrix meets a first condition, determining the first optimized weight matrix as the optimized first weight matrix, or in response to that the difference between the first weight matrix and the first optimized weight matrix does not meet the first condition, obtaining new fitting image features using the first optimized weight matrix, repeating the first optimizing processing based on the new fitting image features until a difference between a k-th optimized weight matrix and a (k−1)th optimized weight matrix meets the first condition and determining the k-th optimized weight matrix as the optimized first weight matrix, wherein k is a positive integer greater than 1.

### Claim 6 (depends on 5)

The method of claim 5, wherein performing the first optimizing processing on the first weight matrix using the first error between the image feature of each of the plurality of images and the fitting image feature of the image comprises:
obtaining the first error between the image feature and the fitting image feature according to a sum of squares of differences between elements in the image feature and corresponding elements in the fitting image feature;
obtaining a second weight coefficient of each image feature based on each first error; and
performing the first optimizing processing on the first weight matrix based on the second weight coefficient of each of the plurality of images to obtain the first optimized weight matrix corresponding to the first weight matrix.

### Claim 7 (depends on 6)

The method of claim 6, wherein obtaining the second weight coefficient of each image feature based on each first error comprises:
obtaining the second weight coefficient of each image feature in a first manner based on each first error, wherein an expression of the first manner is:
w
i
=
{
1
,
⁢
e
i
≤
k
k

e
i

,
⁢
e
i
>
k
,
where is the second weight coefficient of an i-th image; ei represents a first error between an i-th image feature and a fitting image feature corresponding to the i-th image feature, i is an integer between i and N, and N is a number of the image features; and k=1.345 σ, and σ is a standard deviation of the errors ei.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining, according to the image feature of each of the plurality of images, the weight coefficient having the one-to-one correspondence with each image feature further comprises:
forming the image feature matrix based on the image feature of each of the plurality of images;
performing median filtering processing on the image feature matrix to obtain a median feature matrix; and
determining the weight coefficient corresponding to each image feature based on the median feature matrix.

### Claim 9 (depends on 8)

The method of claim 8, wherein performing median filtering processing on the image feature matrix to obtain the median feature matrix comprises:
determining an element median of element values at a same position for each image feature in the image feature matrix; and
obtaining the median feature matrix based on the element median at each position.

### Claim 10 (depends on 8)

The method of claim 8, wherein determining the weight coefficient corresponding to each image feature based on the median feature matrix comprises:
obtaining a second error between each image feature and the median feature matrix; and
in response to that a second error between an image feature and the median feature matrix meets a second condition, configuring the weight coefficient of the image feature to be a first weight value, or in response to that a second error between an image feature and the median feature matrix does not meet the second condition, determining the weight coefficient of the image feature in a second manner.

### Claim 11 (depends on 10)

The method of claim 10, wherein expressions of the second manner are:
b
h
=
θ
h
/
∑
h
=
1
N
⁢
θ
h
,
and
θ
h
=
1
/
e
h
,
where bh is a weight coefficient of an h-th image determined in the second manner; and eh is a second error between an image feature of the h-th image and the median feature matrix, h is an integer between 1 and N, and N represents a number of the plurality of images.

### Claim 12 (depends on 10)

The method of claim 10, wherein the second condition is:
eh>K·MADN, and
MADN=median([e1, e2, . . . , eN])/0.675.
where eh is the second error between the image feature of the h-th image and the median feature matrix, h is an integer between 1 and N, and N represents the number of the plurality of images; K is a determination threshold; and median represents a median filtering function.

### Claim 13 (depends on 1)

The method of claim 1, wherein performing the feature fusion processing on the image features of the plurality of images based on the weight coefficient of each image feature to obtain the fusion feature of the plurality of images comprises:
obtaining the fusion feature by summing products of the image features and corresponding weight coefficients.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
performing a recognition operation on the same object using the fusion feature.

### Claim 15 (depends on 1)

The method of claim 1, further comprising: before determining the weight coefficient corresponding to the image feature of each of the plurality of images according to the image feature,
obtaining selection information for a mode for obtaining the weight coefficients;
determining, based on the selection information, the mode for obtaining the weight coefficients;
performing, based on the determined mode for obtaining the weight coefficients, the determination of the weight coefficient corresponding to the image feature of each of the plurality of images according to the image feature,
wherein the mode for obtaining the weight coefficients comprises obtaining the weight coefficients in a feature fitting manner and obtaining the weight coefficients in a median filtering manner.

### Claim 16

An image processing device, comprising:
a memory storing processor-executable instructions; and
a processor configured to execute the stored processor-executable instructions to perform operations of:
obtaining an image feature of each of a plurality of images for a same object;
determining, according to the image feature of each of the plurality of images, a weight coefficient having a one-to-one correspondence with each image feature; and
performing feature fusion processing on the image features of the plurality of images based on the weight coefficient of each image feature to obtain a fusion feature of the plurality of images.

### Claim 17 (depends on 16)

The device of claim 16, wherein determining, according to the image feature of each of the plurality of images, the weight coefficient having the one-to-one correspondence with each image feature comprises:
forming an image feature matrix based on the image feature of each of the plurality of images;
performing feature fitting processing on the image feature matrix to obtain a first weight matrix; and
determining the weight coefficient corresponding to each image feature based on the first weight matrix.

### Claim 18 (depends on 17)

The device of claim 17, wherein performing the feature fitting processing on the image feature matrix to obtain the first weight matrix comprises:
performing the feature fitting processing on the image feature matrix using a regularized least-square linear regression algorithm, and obtaining the first weight matrix under condition that a preset target function takes a minimum value.

### Claim 19 (depends on 17)

The device of claim 17, wherein determining the weight coefficient corresponding to each image feature based on the first weight matrix comprises:
determining each of first weight coefficients comprised in the first weight matrix as the weight coefficient corresponding to each image feature; or
performing first optimizing processing on the first weight matrix, and determining each of first weight coefficients comprised in an optimized first weight matrix as the weight coefficient corresponding to each image feature.

### Claim 20

A non-transitory computer-readable storage medium having stored thereon computer-readable instructions that, when executed by a processor, cause the processor to perform an image processing method, the method comprising:
obtaining an image feature of each of a plurality of images for a same object;
determining, according to the image feature of each of the plurality of images, a weight coefficient having a one-to-one correspondence with each image feature; and
performing feature fusion processing on the image features of the plurality of images based on the weight coefficient of each image feature to obtain a fusion feature of the plurality of images.
