# US 12,183,111 B2 · Methods of performing real-time object detection using object real-time detection model, performance optimization methods of object real-time detection model, electronic devices and computer readable storage media

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,183,111 B2
- Document ID: US12183111B2
- Status: Granted
- Kind: B2
- Application number: 17/578,252
- Filed: 2022-01-18
- Published: 2022-07-28
- Granted: 2024-12-31
- Earliest claimed: 2021-01-25
- Assignees: Boe Technology Group Co., Ltd.
- Applicants: BOE Technology Group Co., Ltd.
- Inventors: Chunshan Zu
- Web page: https://patentgrep.com/patent/US12183111B2

## CPC

- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V 10/32: Normalisation of the pattern dimensions
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure relates to a method of performing real-time object detection using an object real-time detection model and a performance optimization method of object real-time detection model. According to an embodiment, the method of performing real-time object detection using an object real-time detection model includes: obtaining an identification image of a preset size by pre-processing an input image; obtaining object central point data and object size data by processing the identification image using the object real-time detection model; and obtaining an object detection result by determining an object region in the input image according to the object central point data and the object size data.

## Claims

### Claim 1

A method of performing real-time object detection using an object real-time detection model, comprising:
obtaining an identification image of a preset size by pre-processing an input image;
obtaining object central point data and object size data by processing the identification image using the object real-time detection model, wherein the object central point data represents a position of an object central point in the identification image and the object size data represents a width and a height of an object in the identification image; and
obtaining an object detection result by determining an object region in the input image according to the object central point data and the object size data;
wherein obtaining the object central point data and the object size data by processing the identification image using the object real-time detection model comprises:
extracting a semantic feature map of at least one feature layer relating to the object from the identification image by using a feature extraction network in the object real-time detection model;
obtaining a target feature map by performing up-sampling for the semantic feature map of the at least one feature layer and performing fusion for feature maps obtained by up-sampling using a feature fusion network in the object real-time detection model; and
obtaining the object central point data and the object size data by extracting an object central point and an object size from the target feature map using a data obtaining network in the object real-time detection model;
wherein the feature extraction network comprises a first ShuffleNet network and the method further comprises obtaining the feature extraction network in the object real-time detection model by:
obtaining a trained second ShuffleNet network by training a second ShuffleNet network using a training set;
updating the first ShuffleNet network using one or more network structure parameter values of the trained second ShuffleNet network; and
obtaining a trained first ShuffleNet network by training the updated first ShuffleNet network and taking the trained first ShuffleNet network as the feature extraction network.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the feature extraction network includes 5×5 depthwise separable convolution kernels.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the trained first ShuffleNet network includes 5×5 depthwise separable convolution kernels.

### Claim 4 (depends on 1)

The method according to claim 1, wherein updating the first ShuffleNet network using the one or more network structure parameter values of the trained second ShuffleNet network comprises:
updating parameter values of respective N1×N1 depthwise separable convolution kernels in the first ShuffleNet network to corresponding parameter values of respective N1×N1 depthwise separable convolution kernels in the trained second ShuffleNet network; and
performing zero padding for the respective N1×N1 depthwise separable convolution kernels in the updated first ShuffleNet network to obtain N2×N2 depthwise separable convolution kernels, wherein N1 and N2 are integers greater than 0 and N2 is greater than N1.

### Claim 5 (depends on 1)

The method according to claim 1, wherein a convolution kernel in the feature fusion network of the object real-time detection model is implemented by using a 5×5 depthwise separable convolution kernel.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the data obtaining network in the object real-time detection model is implemented by a feature layer in a CenterNet network and configured to obtain the object central point and the object size, and a convolution kernel in the data obtaining network of the object real-time detection model is implemented using a 3×3 depthwise separable convolution kernel.

### Claim 7 (depends on 1)

A non-transitory computer readable storage medium, wherein executable computer programs in the storage medium are executed by a processor to implement the method according to claim 1.

### Claim 8

An electronic device, comprising:
a processor;
a memory for storing computer programs executable by the processor;
wherein the processor is configured to execute the computer programs in the memory to:
obtain an identification image of a preset size by pre-processing an input image obtain object central point data and object size data by processing the identification image using the object real-time detection model, wherein the object central point data represents a position of an object central point in the identification image and the object size data represents a width and a height of an object in the identification image; and
obtain an object detection result by determining an object region in the input image according to the object central point data and the object size data;
wherein when obtaining the object central point data and the object size data by processing the identification image using the object real-time detection model, the processor is caused to:
extract a semantic feature map of at least one feature layer relating to the object from the identification image by using a feature extraction network in the object real-time detection model;
obtain a target feature map by performing up-sampling for the semantic feature map of the at least one feature layer and performing fusion for feature maps obtained by up-sampling using a feature fusion network in the object real-time detection model; and
obtain the object central point data and the object size data by extracting an object central point and an object size from the target feature map using a data obtaining network in the object real-time detection model;
wherein the feature extraction network comprises a first ShuffleNet network, and the processor is further caused to:
obtain a trained second ShuffleNet network by training a second ShuffleNet network using a training set;
update the first ShuffleNet network using one or more network structure parameter values of the trained second ShuffleNet network; and
obtain a trained first ShuffleNet network by training the updated first ShuffleNet network and take the trained first ShuffleNet network as the feature extraction network.

### Claim 9 (depends on 8)

The electronic device according to claim 8, wherein the trained first ShuffleNet network includes 5×5 depthwise separable convolution kernels.

### Claim 10 (depends on 8)

The electronic device according to claim 8, wherein when updating the first ShuffleNet network using the one or more network structure parameter values of the trained second ShuffleNet network, the processor is caused to:
update parameter values of respective N1×N1 depthwise separable convolution kernels in the first ShuffleNet network to corresponding parameter values of respective N1×N1 depthwise separable convolution kernels in the trained second ShuffleNet network; and
perform zero padding for the respective N1×N1 depthwise separable convolution kernels in the updated first ShuffleNet network to obtain N2×N2 depthwise separable convolution kernels, wherein N1 and N2 are integers greater than 0 and N2 is greater than N1.

### Claim 11 (depends on 8)

The electronic device according to claim 8, wherein a convolution kernel in the feature fusion network of the object real-time detection model is implemented by using a 5×5 depthwise separable convolution kernel.

### Claim 12 (depends on 8)

The electronic device according to claim 8, wherein the data obtaining network in the object real-time detection model is implemented by a feature layer in a CenterNet network and configured to obtain the object central point and the object size, and a convolution kernel in the data obtaining network of the object real-time detection model is implemented using a 3×3 depthwise separable convolution kernel.

### Claim 13 (depends on 8)

The electronic device according to claim 8, wherein the feature extraction network includes 5λ5 depthwise separable convolution kernels.