# US 2025/0095327 A1 · Method, apparatus, readable storage medium, and electronic device for object attribute recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095327 A1
- Document ID: US20250095327A1
- Status: Published application
- Kind: A1
- Application number: 18/730,536
- Filed: 2022-12-26
- Published: 2025-03-20
- Earliest claimed: 2022-01-21
- Assignees: Beijing Youzhuju Network Technology Co., Ltd.
- Applicants: Beijing Youzhuju Network Technology Co., Ltd.
- Inventors: Xiaofei MAO; Can HUANG
- Web page: https://patentgrep.com/patent/US20250095327A1

## CPC

- G06F 18/24: Classification techniques
- G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
- G06F 18/00: Pattern recognition
- G06F 18/253: {of extracted features}
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V 2201/07: Target detection
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The disclosure relates to a method, apparatus, readable storage medium, and electronic device for object attribute recognition. The method includes: acquiring a target image, the target image comprising a target object and object description information of the target object; extracting, from the target image, a sequence of key information features of the target object and a sequence of multimodal features corresponding to a target attribute of the target object, the sequence of multimodal features comprising a sequence of visual features and a sequence of semantic features of the target attribute; and determining a plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features.

## Claims

### Claim 1

A method of object attribute recognition, comprising:
acquiring a target image, wherein the target image comprises a target object and object description information of the target object;
extracting, from the target image, a sequence of key information features of the target object and a sequence of multimodal features corresponding to a target attribute of the target object, wherein the sequence of multimodal features comprises a sequence of visual features and a sequence of semantic features of the target attribute; and
determining a plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features, wherein the plurality of object attributes comprises the target attribute.

### Claim 2 (depends on 1)

The method of claim 1, wherein extracting, from the target image, the sequence of multimodal features corresponding to the target attribute of the target object comprises:
inputting the target image into a pre-trained multimodal feature extraction model to obtain the sequence of multimodal features corresponding to the target attribute of the target object.

### Claim 3 (depends on 2)

The method of claim 2, wherein the multimodal feature extraction model comprises:
a first target detection module configured to extract, from the target image, a first region where an identification of the target attribute of the target object is located;
a first preprocessing module connected to the first target detection module and configured to normalize the first region into an image of a first predetermined size and straighten the normalized image into a one-dimensional row vector of a first predetermined length;
a first fully-connected module connected to the first preprocessing module and configured to generate the sequence of visual features of the target attribute based on the one-dimensional row vector of the first predetermined length;
a text recognition module connected to the first target detection module and configured to perform text recognition of the first region to obtain attribute description text of the target attribute;
a multilingual language sub-model connected to the text recognition module and configured to extract, from the attribute description text, the sequence of semantic features of the target attribute;
a concatenating module connected to the first fully-connected module and the multilingual language sub-model respectively, and configured to concatenate the sequence of visual features and the sequence of semantic features of the target attribute to obtain a concatenated sequence;
a first encoding module connected to the concatenate module and configured to encode the concatenated sequence to obtain a first encoded sequence; and
a second fully-connected module connected to the first encoding module and configured to perform dimensionality reduction on the first encoded sequence to obtain the sequence of multimodal features of a predetermined dimension corresponding to the target attribute.

### Claim 4 (depends on 1)

The method of claim 1, wherein extracting, from the target image, the sequence of key information features of the target object comprises:
performing text recognition on the target image to obtain the recognized text, wherein the recognized text is a multilingual text or a monolingual text; and
inputting the recognized text into a pre-trained multilingual language model to obtain the sequence of key information features of the target object.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining the plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features comprises:
inputting the sequence of key information features and the sequence of multimodal features into a pre-trained multimodal fusion model to obtain the plurality of object attributes of the target object;
wherein the multimodal fusion model comprises:
a second encoding module configured to encode a first feature matrix formed by the sequence of key information features and the sequence of multimodal features to obtain a second encoded sequence, wherein the sequence of key information features and the sequence of multimodal features are both of a predetermined dimension; and
a plurality of first decoding modules corresponding one-to-one to attribute categories of the plurality of object attributes, connected to the second encoding module respectively, and configured to generate an object attribute under the corresponding attribute category based on the second encoded sequence, wherein the attribute categories of the object attributes are different from each other.

### Claim 6 (depends on 1)

The method of claim 1, wherein the method further comprises:
extracting, from the target image, a sequence of appearance features of the target object; and
wherein determining the plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features comprises:
determining the plurality of object attributes of the target object based on the sequence of key information features, the sequence of multimodal features, and the sequence of appearance features.

### Claim 7 (depends on 6)

The method of claim 6, wherein extracting, from the target image, a sequence of appearance features of the target object comprises:
inputting the target image into a pre-trained appearance feature extraction model to obtain the sequence of appearance features of the target object, the appearance feature extraction model comprising a second target detection module, a second preprocessing module, a third encoding module, and a third fully-connected module connected in sequence;
wherein the second target detection module is configured to extract, from the target image, a second region where an appearance of the target object is located;
the second preprocessing module is configured to normalize the second region into an image of a second predetermined size and straighten the normalized image into a one-dimensional row vector of a second predetermined length;
the third encoding module is configured to encode the one-dimensional row vector of the second predetermined length to obtain a third encoded sequence; and
the third fully-connected module is configured to perform dimensionality reduction on the third encoded sequence to obtain the sequence of appearance features of the target object with a predetermined dimension.

### Claim 8 (depends on 6)

The method of claim 6, wherein the method further comprises:
extracting, from the target image, a sequence of global visual features of the target image; and
wherein determining the plurality of object attributes of the target object based on the sequence of key information features, the sequence of multimodal features, and the sequence of appearance features comprises:
determining the plurality of object attributes of the target object based on the sequence of key information features, the sequence of multimodal features, the sequence of appearance features, and the sequence of global visual features.

### Claim 9 (depends on 8)

The method of claim 8, wherein extracting, from the target image, the sequence of global visual features of the target object comprises:
inputting the target image into a pre-trained global visual feature extraction model to obtain the sequence of global visual features of the target image, the global visual feature extraction model comprising a third preprocessing module, a fourth fully-connected module, a fourth encoding module, and a second decoding module connected in sequence;
wherein the third preprocessing module is configured to adjust the target image to be of a third predetermined size, partition the size-adjusted target image into a plurality of image blocks according to a fourth predetermined size, and thereafter, straighten each of the image blocks into a one-dimensional feature vector of a third predetermined length, and form a second feature matrix from each of the one-dimensional feature vectors of the third predetermined length;
the fourth fully-connected module is configured to generate, based on the second feature matrix, a sequence of original features corresponding to the target image;
the fourth encoding module is configured to encode the sequence of original features to obtain a fourth encoded sequence; and
the second decoding module is configured to decode the fourth encoded sequence to obtain the sequence of global visual features of the target image.

### Claim 10

(canceled)

### Claim 11

A non-transitory computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processing device, implements acts comprising:
acquiring a target image, wherein the target image comprises a target object and object description information of the target object;
extracting, from the target image, a sequence of key information features of the target object and a sequence of multimodal features corresponding to a target attribute of the target object, wherein the sequence of multimodal features comprises a sequence of visual features and a sequence of semantic features of the target attribute; and
determining a plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features, wherein the plurality of object attributes comprises the target attribute.

### Claim 12

An electronic device, comprising:
a storage device having a computer program stored thereon; and
a processing device configured to execute the computer program in the storage device to implement the acts comprising:
acquiring a target image, wherein the target image comprises a target object and object description information of the target object;
extracting, from the target image, a sequence of key information features of the target object and a sequence of multimodal features corresponding to a target attribute of the target object, wherein the sequence of multimodal features comprises a sequence of visual features and a sequence of semantic features of the target attribute; and
determining a plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features, wherein the plurality of object attributes comprises the target attribute.

### Claim 13 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein extracting, from the target image, the sequence of multimodal features corresponding to the target attribute of the target object comprises:
inputting the target image into a pre-trained multimodal feature extraction model to obtain the sequence of multimodal features corresponding to the target attribute of the target object.

### Claim 14 (depends on 13)

The non-transitory computer-readable medium of claim 13, wherein the multimodal feature extraction model comprises:
a first target detection module configured to extract, from the target image, a first region where an identification of the target attribute of the target object is located;
a first preprocessing module connected to the first target detection module and configured to normalize the first region into an image of a first predetermined size and straighten the normalized image into a one-dimensional row vector of a first predetermined length;
a first fully-connected module connected to the first preprocessing module and configured to generate the sequence of visual features of the target attribute based on the one-dimensional row vector of the first predetermined length;
a text recognition module connected to the first target detection module and configured to perform text recognition of the first region to obtain attribute description text of the target attribute;
a multilingual language sub-model connected to the text recognition module and configured to extract, from the attribute description text, the sequence of semantic features of the target attribute;
a concatenating module connected to the first fully-connected module and the multilingual language sub-model respectively, and configured to concatenate the sequence of visual features and the sequence of semantic features of the target attribute to obtain a concatenated sequence;
a first encoding module connected to the concatenate module and configured to encode the concatenated sequence to obtain a first encoded sequence; and
a second fully-connected module connected to the first encoding module and configured to perform dimensionality reduction on the first encoded sequence to obtain the sequence of multimodal features of a predetermined dimension corresponding to the target attribute.

### Claim 15 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein extracting, from the target image, the sequence of key information features of the target object comprises:
performing text recognition on the target image to obtain the recognized text, wherein the recognized text is a multilingual text or a monolingual text; and
inputting the recognized text into a pre-trained multilingual language model to obtain the sequence of key information features of the target object.

### Claim 16 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein determining the plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features comprises:
inputting the sequence of key information features and the sequence of multimodal features into a pre-trained multimodal fusion model to obtain the plurality of object attributes of the target object;
wherein the multimodal fusion model comprises:
a second encoding module configured to encode a first feature matrix formed by the sequence of key information features and the sequence of multimodal features to obtain a second encoded sequence, wherein the sequence of key information features and the sequence of multimodal features are both of a predetermined dimension; and
a plurality of first decoding modules corresponding one-to-one to attribute categories of the plurality of object attributes, connected to the second encoding module respectively, and configured to generate an object attribute under the corresponding attribute category based on the second encoded sequence, wherein the attribute categories of the object attributes are different from each other.

### Claim 17 (depends on 11)

The non-transitory computer-readable medium of claim 11, wherein the acts further comprises:
extracting, from the target image, a sequence of appearance features of the target object; and
wherein determining the plurality of object attributes of the target object based on the sequence of key information features and the sequence of multimodal features comprises:
determining the plurality of object attributes of the target object based on the sequence of key information features, the sequence of multimodal features, and the sequence of appearance features.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein extracting, from the target image, a sequence of appearance features of the target object comprises:
inputting the target image into a pre-trained appearance feature extraction model to obtain the sequence of appearance features of the target object, the appearance feature extraction model comprising a second target detection module, a second preprocessing module, a third encoding module, and a third fully-connected module connected in sequence;
wherein the second target detection module is configured to extract, from the target image, a second region where an appearance of the target object is located;
the second preprocessing module is configured to normalize the second region into an image of a second predetermined size and straighten the normalized image into a one-dimensional row vector of a second predetermined length;
the third encoding module is configured to encode the one-dimensional row vector of the second predetermined length to obtain a third encoded sequence; and
the third fully-connected module is configured to perform dimensionality reduction on the third encoded sequence to obtain the sequence of appearance features of the target object with a predetermined dimension.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the acts further comprises:
extracting, from the target image, a sequence of global visual features of the target image; and
wherein determining the plurality of object attributes of the target object based on the sequence of key information features, the sequence of multimodal features, and the sequence of appearance features comprises:
determining the plurality of object attributes of the target object based on the sequence of key information features, the sequence of multimodal features, the sequence of appearance features, and the sequence of global visual features.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein extracting, from the target image, the sequence of global visual features of the target object comprises:
inputting the target image into a pre-trained global visual feature extraction model to obtain the sequence of global visual features of the target image, the global visual feature extraction model comprising a third preprocessing module, a fourth fully-connected module, a fourth encoding module, and a second decoding module connected in sequence;
wherein the third preprocessing module is configured to adjust the target image to be of a third predetermined size, partition the size-adjusted target image into a plurality of image blocks according to a fourth predetermined size, and thereafter, straighten each of the image blocks into a one-dimensional feature vector of a third predetermined length, and form a second feature matrix from each of the one-dimensional feature vectors of the third predetermined length;
the fourth fully-connected module is configured to generate, based on the second feature matrix, a sequence of original features corresponding to the target image;
the fourth encoding module is configured to encode the sequence of original features to obtain a fourth encoded sequence; and
the second decoding module is configured to decode the fourth encoded sequence to obtain the sequence of global visual features of the target image.

### Claim 21 (depends on 12)

The device of claim 12, wherein extracting, from the target image, the sequence of multimodal features corresponding to the target attribute of the target object comprises:
inputting the target image into a pre-trained multimodal feature extraction model to obtain the sequence of multimodal features corresponding to the target attribute of the target object.