# US 2024/0005646 A1 · Method for generating saliency map, and method and apparatus for detecting abnormal object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005646 A1
- Document ID: US20240005646A1
- Status: Published application
- Kind: A1
- Application number: 18/368,120
- Filed: 2023-09-14
- Published: 2024-01-04
- Earliest claimed: 2021-03-15
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Yushun XU; Yi YANG; Jing LIU; Luyu QIU
- Web page: https://patentgrep.com/patent/US20240005646A1

## CPC

- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/82: using neural networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06T 7/0012: {Biomedical image inspection}
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/776: Validation; Performance evaluation
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30004: Biomedical image processing
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments of this application relate to the field of artificial intelligence, and disclose a method for generating a saliency map. The method includes: performing disturbance processing on a first object to obtain a plurality of objects; screening the plurality of objects to obtain a plurality of updated objects, where the plurality of updated objects satisfy target data distribution, and a training sample of a target model also satisfies the target data distribution; inputting the plurality of updated objects into the target model to output a first prediction result; and generating a saliency map of the first object based on the first prediction result and the plurality of updated objects.

## Claims

### Claim 1

A method for generating a saliency map, comprising:
obtaining a plurality of objects, wherein the plurality of objects are obtained by performing disturbance processing on a first object;
performing screening processing on the plurality of objects based on a first condition, to obtain a plurality of updated objects, wherein the plurality of updated objects satisfy target data distribution, the target data distribution is obtained based on a training sample, and the training sample is used to train a preset model to obtain a target model;
obtaining an input of the target model based on the plurality of updated objects; and
generating a saliency map of the first object based on a first prediction result output by the target model and the plurality of updated objects.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the first condition is deleting a target object from the plurality of objects, a distance between a feature of the target object and a weight vector of the target model exceeds a preset threshold, and the feature of the target object is obtained by performing feature extraction on the target object by using the target model.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the feature of the target object is extracted by using a first feature extraction layer, the first feature extraction layer is any one of a plurality of feature extraction layers in the target model, the distance between the feature of the target object and the weight vector of the target model is a distance between the feature of the target object and a weight vector of a second feature extraction layer, and the second feature extraction layer is any one of the plurality of feature extraction layers.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the first feature extraction layer and the second feature extraction layer are different feature extraction layers.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the first feature extraction layer is a penultimate feature extraction layer in a plurality of feature extraction layers in a feature extraction model, the plurality of feature extraction layers are connected end-to-end, the target object is an input of the first feature extraction layer in the plurality of feature extraction layers, and the second feature extraction layer is a last feature extraction layer in the plurality of feature extraction layers.

### Claim 6 (depends on 3)

The method according to claim 3, wherein a distance between the feature of the target object and a target weight vector of the second feature extraction layer exceeds the preset threshold, the distance between the feature of the target object and the target weight vector of the second feature extraction layer is a longest distance in a plurality of distances, and the plurality of distances comprise a distance between the feature of the target object and each weight vector of the second feature extraction layer.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the target model is obtained by updating the preset model by using a first loss value, the first loss value is determined based on a deviation between a feature of the training sample and a weight vector of the preset model, and the feature of the training sample is obtained by performing feature extraction on the training sample by using the preset model.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the target model is obtained by updating the preset model by using the first loss value and a second loss value, the second loss value is determined based on a deviation between a target result and a real result of the training sample, the target result is determined based on a second prediction result and a preset function, the second prediction result is a prediction result of the preset model for the training sample, an input of the preset function is the second prediction result, an output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
setting weights of the plurality of updated objects to a first weight; and
setting weights of a plurality of remaining objects to a second weight, wherein the plurality of remaining objects are objects other than the plurality of updated objects in the plurality of objects, and the first weight is greater than the second weight; and wherein
the obtaining an input of the target model based on the plurality of updated objects comprises:
obtaining a first result based on the first weight and the plurality of updated objects, wherein the first result is an input of the target model, the input of the target model further comprises a second result, and the second result is obtained based on the second weight and the plurality of remaining objects.

### Claim 10

A method for detecting an abnormal object, comprising:
obtaining a plurality of objects;
performing feature extraction on a target object by using a feature extraction model, to obtain a feature of the target object, wherein the target object is any one of the plurality of objects, the feature extraction model is obtained by updating a preset model by using a first loss value, the first loss value is determined based on a deviation between a feature of a training sample and a weight vector of the preset model, and the feature of the training sample is obtained by performing feature extraction on the training sample by using the preset model; and
obtaining a distance between the feature of the target object and a weight vector of the feature extraction model, wherein when the distance exceeds a preset threshold, the target object is an abnormal object.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the performing feature extraction on a target object by using a feature extraction model, to obtain a feature of the target object comprises:
performing feature extraction on the target object by using a first feature extraction layer, to obtain the feature of the target object, wherein the first feature extraction layer is any one of a plurality of feature extraction layers in the feature extraction model; and wherein
the obtaining a distance between the feature of the target object and a weight vector of the feature extraction model comprises:
obtaining a distance between the feature of the target object and a weight vector of a second feature extraction layer, wherein the second feature extraction layer is any one of the plurality of feature extraction layers.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the first feature extraction layer and the second feature extraction layer are different feature extraction layers.

### Claim 13 (depends on 11)

The method according to claim 11, wherein the first feature extraction layer is a penultimate feature extraction layer in the plurality of feature extraction layers in the feature extraction model, the plurality of feature extraction layers are connected end-to-end, the target object is an input of the first feature extraction layer in the plurality of feature extraction layers, and the second feature extraction layer is a last feature extraction layer in the plurality of feature extraction layers.

### Claim 14 (depends on 11)

The method according to claim 11, wherein there are a plurality of weight vectors of the second feature extraction layer, and the obtaining a distance between the feature of the target object and a weight vector of a second feature extraction layer comprises:
obtaining a distance between the feature of the target object and each weight vector of the second feature extraction layer, wherein when a longest distance in a plurality of distances exceeds the preset threshold, the target object is an abnormal sample.

### Claim 15 (depends on 10)

The method according to claim 10, wherein the feature extraction model is obtained by updating the preset model by using the first loss value and a second loss value, the second loss value is determined based on a deviation between a target result and a real result of the training sample, the target result is determined based on a first prediction result and a preset function, the first prediction result is a prediction result of the preset model for the training sample, an input of the preset function is the first prediction result, an output of the preset function is the target result, and the output of the preset function is negatively correlated with the input of the preset function.

### Claim 16 (depends on 10)

The method according to claim 10, further comprising:
deleting the target object from the plurality of objects to obtain a plurality of updated objects, wherein the plurality of updated objects are used to obtain a saliency map of a first image, wherein the plurality of objects are obtained by performing disturbance processing on the first image.

### Claim 17

An electronic device, wherein the electronic device comprises:
at least one processor; and
at least one memory coupled to the at least one processor to store program instructions, which when executed by the processor, cause the at least one processor to:
obtain a plurality of objects, wherein the plurality of objects are obtained by performing disturbance processing on a first object;
perform screening processing on the plurality of objects based on a first condition, to obtain a plurality of updated objects, wherein the plurality of updated objects satisfy target data distribution, the target data distribution is obtained based on a training sample, and the training sample is used to train a preset model to obtain a target model;
obtain an input of the target model based on the plurality of updated objects; and
generate a saliency map of the first object based on a first prediction result output by the target model and the plurality of updated objects.

### Claim 18 (depends on 17)

The device according to claim 17, wherein the first condition is deleting a target object from the plurality of objects, a distance between a feature of the target object and a weight vector of the target model exceeds a preset threshold, and the feature of the target object is obtained by performing feature extraction on the target object by using the target model.

### Claim 19 (depends on 17)

The device according to claim 17, wherein a feature of a target object is extracted by using a first feature extraction layer, the first feature extraction layer is any one of a plurality of feature extraction layers in the target model, a distance between the feature of the target object and a weight vector of the target model is a distance between the feature of the target object and a weight vector of a second feature extraction layer, and the second feature extraction layer is any one of the plurality of feature extraction layers.

### Claim 20 (depends on 19)

The device according to claim 19, wherein the first feature extraction layer and the second feature extraction layer are different feature extraction layers.
