# US 2021/0279500 A1 · Method and apparatus for identifying key point locations in image, and medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279500 A1
- Document ID: US20210279500A1
- Status: Published application
- Kind: A1
- Application number: 17/212,635
- Filed: 2021-03-25
- Published: 2021-09-09
- Earliest claimed: 2020-05-29
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science and Technology Co., Ltd.
- Inventors: Xuetong Xue; Hui Ren; Min Yang
- Web page: https://patentgrep.com/patent/US20210279500A1

## CPC

- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure provides a method for identifying key point locations in an image. The method includes: obtaining an image to be detected, and obtaining a feature map and a saliency map of the image to be detected; generating a score response map of the image to be detected based on the feature map of the image to be detected; obtaining a first key point location of a key point on the score response map; obtaining a second key point location of the key point mapped on the image to be detected based on the first key point location; and correcting the second key point location based on the saliency map.

## Claims

### Claim 1

A method for identifying key point locations in an image, comprising:
obtaining an image to be detected, and obtaining a feature map and a saliency map of the image to be detected;
generating a score response map of the image to be detected based on the feature map of the image to be detected;
obtaining a first key point location of a key point on the score response map;
obtaining a second key point location of the key point mapped on the image to be detected based on the first key point location; and
correcting the second key point location based on the saliency map.

### Claim 2 (depends on 1)

The method according to claim 1, wherein generating the score response map of the image to be detected based on the feature map of the image to be detected comprises:
obtaining local response scores of feature points on the feature map;
obtaining channel response scores of the feature points on the feature map;
calculating response scores of the feature points based on the local response scores and the channel response scores; and
generating the score response map of the image to be detected based on the response scores of the feature points.

### Claim 3 (depends on 2)

The method according to claim 2, wherein obtaining the local response scores of the feature points on the feature map comprises:
obtaining a plurality of feature points adjacent to an ith feature point, i being a positive integer;
obtaining a feature value of the ith feature point and feature values of the plurality of feature points adjacent to the ith feature point; and
generating a local response score of the ith feature point based on the feature value of the ith feature point and the feature values of the plurality of feature points adjacent to the ith feature point, and sequentially traversing the feature points on the feature map to obtain the local response scores of the feature points.

### Claim 4 (depends on 3)

The method according to claim 3, wherein obtaining the channel response scores of the feature points on the feature map comprises:
obtaining a plurality of feature values of the ith feature point on a plurality of channels;
generating a channel response score of the ith feature point based on the plurality of feature values of the ith feature point on the plurality of channels; and sequentially traversing the feature points on the feature map to obtain the channel response scores of the feature points.

### Claim 5 (depends on 2)

The method according to claim 2, wherein calculating the response scores of the feature points based on the local response scores and the channel response scores, comprises:
calculating a response score of each feature point by multiplying a local response score of the feature point with a channel response score of the feature point.

### Claim 6 (depends on 1)

The method according to claim 1, wherein obtaining the first key point location of the key point on the score response map comprises:
obtaining response scores of feature points on the score response map; and
determining a feature point with a response score greater than a preset threshold as the key point, and obtaining the first key point location of the key point.

### Claim 7 (depends on 1)

The method according to claim 1, wherein a resolution of the feature map is smaller than a resolution of the image to be detected.

### Claim 8 (depends on 7)

The method according to claim 7, wherein obtaining the second key point location of the key point mapped on the image to be detected based on the first key point location, comprises:
obtaining a ratio of the resolution of the feature map to the resolution of the image to be detected; and
generating the second key point location based on the ratio and the first key point location.

### Claim 9 (depends on 1)

The method according to claim 1, wherein correcting the second key point location based on the saliency map comprises:
obtaining an area comprising the second key point location;
obtaining saliency values of feature points in the area;
obtaining a location of a feature point with the largest saliency value in the area, and determining the location of the feature point with the largest saliency value in the area as a corrected location; and
correcting the second key point location based on the corrected position.

### Claim 10

An apparatus for identifying key point locations in an image, comprising:
one or more processors;
a memory storing instructions executable by the one or more processors;
wherein the one or more processors are configured to:
obtain an image to be detected, and to obtain a feature map and a saliency map of the image to be detected;
generate a score response map of the image to be detected based on the feature map of the image to be detected;
obtain a first key point location of a key point on the score response map;
obtain a second key point location of the key point mapped on the image to be detected based on the first key point location; and
correct the second key point location based on the saliency map.

### Claim 11 (depends on 10)

The apparatus according to claim 10, wherein the one or more processors are configured to:
obtain local response scores of feature points on the feature map;
obtain channel response scores of the feature points on the feature map;
calculate response scores of the feature points based on the local response scores and the channel response scores; and
generate the score response map of the image to be detected based on the response scores of the feature points.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein the one or more processors are configured to obtain a plurality of feature points adjacent to an ith feature point, to obtain a feature value of the ith feature point and feature values of the plurality of feature points adjacent to the ith feature point, to generate a local response score of the feature point based on the ith feature value of the ith feature point and the feature values of the plurality of feature points adjacent to the ith feature point, and to sequentially traverse the feature points on the feature map to obtain the local response scores of the feature points, in which i is a positive integer.

### Claim 13 (depends on 12)

The apparatus according to claim 12, wherein the one or more processors are configured to obtain a plurality of feature values of the ith feature point on a plurality of channels, to generate a channel response score of the ith feature point based on the plurality of feature values of the ith feature point on the plurality of channels, and to sequentially traverse the feature points on the feature map to obtain the channel response scores of the feature points.

### Claim 14 (depends on 11)

The apparatus according to claim 11, wherein the one or more processors are configured to calculate a response score of each feature point by multiplying a local response score of the feature point with a channel response score of the feature point.

### Claim 15 (depends on 10)

The apparatus according to claim 10, wherein the one or more processors are configured to:
obtain response scores of feature points on the score response map; and
determine a feature point with a response score greater than a preset threshold as the key point, and to obtain the first key point location of the key point.

### Claim 16 (depends on 10)

The apparatus according to claim 10, wherein a resolution of the feature map is smaller than a resolution of the image to be detected.

### Claim 17 (depends on 16)

The apparatus according to claim 16, wherein the one or more processors are configured to:
obtain a ratio of the resolution of the feature map to the resolution of the image to be detected; and
generate the second key point location based on the ratio and the first key point location.

### Claim 18 (depends on 10)

The apparatus according to claim 10, wherein the one or more processors are configured to:
obtain an area comprising the second key point location;
obtain saliency values of feature points in the area;
obtain a location of a feature point with the largest saliency value in the area, and to determine the location of the feature point with the largest saliency value in the area as a corrected location; and
correct the second key point location based on the corrected position.

### Claim 19

A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to implement the method for identifying key point locations in an image, and the method comprise:
obtaining an image to be detected, and obtaining a feature map and a saliency map of the image to be detected;
generating a score response map of the image to be detected based on the feature map of the image to be detected;
obtaining a first key point location of a key point on the score response map;
obtaining a second key point location of the key point mapped on the image to be detected based on the first key point location; and
correcting the second key point location based on the saliency map.
