# US 2022/0036141 A1 · Target object identification method and apparatus

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036141 A1
- Document ID: US20220036141A1
- Status: Published application
- Kind: A1
- Application number: 17/343,154
- Filed: 2021-06-09
- Published: 2022-02-03
- Earliest claimed: 2020-08-01
- Assignees: Sensetime International Pte. Ltd.
- Applicants: Sensetime International Pte. Ltd.
- Inventors: Maoqing TIAN; Jin WU; Shuai YI
- Web page: https://patentgrep.com/patent/US20220036141A1

## CPC

- G06F 18/24147: {Distances to closest patterns, e.g. nearest neighbour classification}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06V 10/24: Aligning, centring, orientation detection or correction of the image
- G06V 10/32: Normalisation of the pattern dimensions
- G06V 10/40: Extraction of image or video features
- G06V 10/82: using neural networks
- G06V 20/00: Scenes; Scene-specific elements (control of digital cameras H04N23/60)
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, devices, systems, and apparatus for target object identification are provided. In one aspect, a method includes: performing classification on a to-be-identified target object in a target image to determine a prediction category of the to-be-identified target object, determining whether the prediction category is correct according to a hidden layer feature for the to-be-identified target object, and outputting prompt information in response to the prediction category being incorrect.

## Claims

### Claim 1

A method of target object identification, the method comprising:
performing classification on a to-be-identified target object in a target image to determine a prediction category of the to-be-identified target object;
determining whether the prediction category is correct according to a hidden layer feature for the to-be-identified target object; and
outputting prompt information in response to determining that the prediction category is incorrect.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
in response to determining that the prediction category is correct, determining the prediction category as a final category of the to-be-identified target object; and
outputting the final category of the to-be-identified target object.

### Claim 3 (depends on 1)

The method according to claim 1, wherein determining whether the prediction category is correct according to the hidden layer feature of the to-be-identified target object comprises:
inputting the hidden layer feature for the to-be-identified target object into an authenticity identification model corresponding to the prediction category, such that the authenticity identification model outputs a probability value, wherein the authenticity identification model corresponding to the prediction category reflects a distribution of hidden layer features for target objects belonging to the prediction category, and the probability value represents a probability that a final category of the to-be-identified target object is the prediction category;
determining that the prediction category is incorrect if the probability value is less than a probability threshold; and
determining that the prediction category is correct if the probability value is greater than or equal to the probability threshold.

### Claim 4 (depends on 3)

The method according to claim 3, wherein performing classification on the to-be-identified target object in the target image is executed by a neural network,
wherein the authenticity identification model corresponding to the prediction category is created by using hidden layer features for authenticated target objects belonging to the prediction category, and
wherein the authenticated target objects are correctly predicted in at least one of a training stage or a test stage of the neural network.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the target image comprises multiple stacked to-be-identified target objects, and
wherein performing classification on the to-be-identified target object in the target image to determine the prediction category of the to-be-identified target object comprises:
adjusting a height of the target image to a preset height, wherein the target image is obtained by cropping an acquired image according to a bounding box of the multiple stacked to-be-identified target objects in the acquired image, and wherein a height direction of the target image is a stacking direction of the multiple stacked to-be-identified target objects; and
performing classification on the to-be-identified target object in the adjusted target image to determine the prediction category of the to-be-identified target object.

### Claim 6 (depends on 5)

The method according to claim 5, wherein adjusting the height of the target image to the preset height comprises:
scaling the height and a width of the target image in equal proportions until the width of the target image reaches a preset width; and
in response to determining that the width of the scaled target image reaches the preset width and the height of the scaled target image is greater than the preset height, reducing the height and the width of the scaled target image in equal proportions until the height of the reduced target image is equal to the preset height.

### Claim 7 (depends on 5)

The method according to claim 5, wherein adjusting the height of the target image to the preset height comprises:
scaling the height and a width of the target image in equal proportions until the width of the target image reaches a preset width; and
in response to determining that the width of the scaled target image reaches the preset width and the height of the scaled target image is less than the preset height, filling the scaled target image with a first pixel, such that the height of the filled scaled target image is equal to the preset height.

### Claim 8 (depends on 5)

The method according to claim 5, wherein performing classification on the to-be-identified target object in the adjusted target image to determine the prediction category of the to-be-identified target object comprises:
performing feature extraction on the adjusted target image to obtain a feature map, wherein a height dimension of the feature map corresponds to the height direction of the target image;
performing average pooling on the feature map in a width dimension of the feature map to obtain a pooled feature map;
segmenting the pooled feature map in the height dimension to obtain a preset number of features; and
determining the prediction category of each of the multiple stacked to-be-identified target objects according to each of the features.

### Claim 9 (depends on 8)

The method according to claim 8, wherein performing classification on the to-be-identified target object in the adjusted target image to determine the prediction category of the to-be-identified target object is executed by a neural network that comprises a classification network, wherein the classification network comprises K classifiers, K being a number of known categories when classifying, K being a positive integer; and
determining the prediction category of each of the multiple stacked to-be-identified target objects according to each of the features comprises:
respectively calculating cosine similarities between each of the features and a weight vector of each of the K classifiers; and
determining the prediction category of each of the multiple stacked to-be-identified target objects according to the calculated cosine similarities.

### Claim 10 (depends on 8)

The method according to claim 8, wherein performing classification on the to-be-identified target object in the adjusted target image to determine the prediction category of the to-be-identified target object is executed by a neural network that comprises a feature extraction network, wherein the feature extraction network comprises multiple convolutional layers, and a corresponding stride of last N convolutional layers of the multiple convolutional layers in the feature extraction network is 1 in the height dimension of the feature map, N being a positive integer.

### Claim 11

An electronic device, comprising:
at least one processor; and
one or more memories coupled to the at least one processor and storing programming instructions for execution by the at least one processor to perform operations comprising:
performing classification on a to-be-identified target object in a target image to determine a prediction category of the to-be-identified target object;
determining whether the prediction category is correct according to a hidden layer feature for the to-be-identified target object; and
outputting prompt information in response to determining that the prediction category is incorrect.

### Claim 12 (depends on 11)

The electronic device according to claim 11, wherein the operations further comprise:
in response to determining that the prediction category is correct, determining the prediction category as a final category of the to-be-identified target object; and
outputting the final category of the to-be-identified target object.

### Claim 13 (depends on 11)

The electronic device according to claim 11, wherein determining whether the prediction category is correct according to the hidden layer feature of the to-be-identified target object comprises:
inputting the hidden layer feature for the to-be-identified target object into an authenticity identification model corresponding to the prediction category, such that the authenticity identification model outputs a probability value, wherein the authenticity identification model corresponding to the prediction category reflects distribution of hidden layer features for target objects belonging to the prediction category, and the probability value represents a probability that a final category of the to-be-identified target object is the prediction category;
determining that the prediction category is incorrect if the probability value is less than a probability threshold; and
determining that the prediction category is correct if the probability value is greater than or equal to the probability threshold.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein performing classification on the to-be-identified target object in the target image is executed by a neural network,
wherein the authenticity identification model corresponding to the prediction category is created by using hidden layer features for authenticated target objects belonging to the prediction category, and
wherein the authenticated target objects are correctly predicted in at least one of a training stage or a test stage of the neural network.

### Claim 15 (depends on 11)

The electronic device according to claim 11, wherein the target image comprises multiple stacked to-be-identified target objects;
performing classification on the to-be-identified target object in the target image to determine the prediction category of the to-be-identified target object comprises:
adjusting a height of the target image to a preset height, wherein the target image is obtained by cropping an acquired image according to a bounding box of the multiple stacked to-be-identified target objects in the acquired image, and wherein a height direction of the target image is a stacking direction of the multiple stacked to-be-identified target objects; and
performing classification on the to-be-identified target object in the adjusted target image to determine the prediction category of the to-be-identified target object.

### Claim 16 (depends on 15)

The electronic device according to claim 15, wherein adjusting the height of the target image to the preset height comprises:
scaling the height and a width of the target image in equal proportions, until the width of the target image reaches a preset width; and
in response to determining that the width of the scaled target image reaches the preset width, and the height of the scaled target image is greater than the preset height, reducing the height and the width of the scaled target image in equal proportions, until the height of the reduced target image is equal to the preset height.

### Claim 17 (depends on 15)

The electronic device according to claim 15, wherein adjusting the height of the target image to the preset height comprises:
scaling the height and a width of the target image in equal proportions, until the width of the target image reaches a preset width; and
in response to determining that the width of the scaled target image reaches the preset width, and the height of the scaled target image is less than the preset height, filling the scaled target image with a first pixel, such that the height of the filled scaled target image is equal to the preset height.

### Claim 18 (depends on 15)

The electronic device according to claim 15, wherein performing classification on the to-be-identified target object in the adjusted target image to determine the prediction category of the to-be-identified target object comprises:
performing feature extraction on the adjusted target image to obtain a feature map, wherein a height dimension of the feature map corresponds to the height direction of the target image;
performing average pooling on the feature map in a width dimension of the feature map to obtain a pooled feature map;
segmenting the pooled feature map in the height dimension to obtain a preset number of features; and
determining the prediction category of each of the multiple stacked to-be-identified target objects according to each of the features.

### Claim 19 (depends on 18)

The electronic device according to claim 18, wherein performing classification on the to-be-identified target object in the adjusted target image to determine the prediction category of the to-be-identified target object is executed by a neural network that comprises a classification network, and wherein the classification network comprises K classifiers, K being a number of known categories when classifying, K being a positive integer; and
wherein determining the prediction category of each of the multiple stacked to-be-identified target objects according to each of the features comprises:
respectively calculating cosine similarities between each of the features and a weight vector of each of the K classifiers; and
determining the prediction category of each of the multiple stacked to-be-identified target objects according to the calculated cosine similarities.

### Claim 20

A non-transitory computer-readable storage medium coupled to the at least one processor and storing programming instructions for execution by the at least one processor to perform operations comprising:
performing classification on a to-be-identified target object in a target image to determine a prediction category of the to-be-identified target object;
determining whether the prediction category is correct according to a hidden layer feature for the to-be-identified target object; and
outputting prompt information in response to determining that the prediction category is incorrect.
