# US 2022/0027606 A1 · Human behavior recognition method, device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0027606 A1
- Document ID: US20220027606A1
- Status: Published application
- Kind: A1
- Application number: 17/494,724
- Filed: 2021-10-05
- Published: 2022-01-27
- Earliest claimed: 2021-01-25
- Assignees: Beijing Baidu Netcom Science Technology Co., Ltd.
- Applicants: Beijing Baidu Netcom Science Technology Co., Ltd.
- Inventors: Tao HU; Xiangbo SU
- Web page: https://patentgrep.com/patent/US20220027606A1

## CPC

- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V 10/40: Extraction of image or video features
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A human behavior recognition method, a device, and a storage medium are provided, which are related to the field of artificial intelligence, specifically to computer vision and deep learning technologies, and applicable to smart city scenarios. The method includes: obtaining attribute information of a target object and N pieces of candidate behavior-related information of a target human from a target image, wherein N is an integer greater than or equal to 1; determining target behavior-related information based on comparison results between the N pieces of candidate behavior-related information and the attribute information of the target object; and determining a behavior recognition result of the target human based on the target behavior-related information.

## Claims

### Claim 1

A human behavior recognition method, comprising:
obtaining attribute information of a target object and N pieces of candidate behavior-related information of a target human from a target image, wherein N is an integer greater than or equal to 1;
determining target behavior-related information based on comparison results between the N pieces of candidate behavior-related information and the attribute information of the target object; and
determining a behavior recognition result of the target human based on the target behavior-related information.

### Claim 2 (depends on 1)

The method according to claim 1, wherein determining the target behavior-related information based on the comparison results between the N pieces of candidate behavior-related information and the attribute information of the target object comprises:
determining a comparison sequence of the N pieces of candidate behavior-related information based on confidence scores respectively corresponding to the N pieces of candidate behavior-related information;
selecting, based on the comparison sequence, a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information, for comparison with the attribute information of the target object; and
taking the piece of candidate behavior to be compared-related information as the target behavior-related information, in a case that a comparison result is that the piece of candidate behavior to be compared-related information matches the attribute information of the target object.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising at least one of the following:
determining, in a case that a category of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information matches a category of the target object, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object; and
determining, in a case that a difference between location information of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information and location information of the target object is within a preset range, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object.

### Claim 4 (depends on 2)

The method according to claim 2, further comprising at least one of the following:
determining, in a case that a category of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information matches a category of the target object, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object; and
determining, in a case that a difference between location information of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information and location information of the target object is within a preset range, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object.

### Claim 5 (depends on 1)

The method according to claim 1, wherein determining the behavior recognition result of the target human based on the target behavior-related information comprises:
taking a category of a target behavior in the target behavior-related information as the behavior recognition result of the target human.

### Claim 6 (depends on 1)

The method according to claim 1, wherein determining the target behavior-related information based on the comparison results between the N pieces of candidate behavior-related information and the attribute information of the target object comprises:
determining basic behavior-related information in the N pieces of candidate behavior-related information as the target behavior-related information, in a case that the comparison results indicate that all the N pieces of candidate behavior-related information do not match the attribute information of the target object.

### Claim 7 (depends on 1)

The method according to claim 1, wherein obtaining the N pieces of candidate behavior-related information of the target human from the target image comprises:
obtaining human image information of the target human from the target image; and
inputting the human image information into a classification model, to obtain the N pieces of candidate behavior-related information of the target human output by the classification model, wherein
a piece of candidate behavior-related information comprises at least one of a category and location information of a candidate behavior.

### Claim 8 (depends on 7)

The method according to claim 7, further comprising:
extracting an image feature of the target image;
determining, based on the image feature, the target human, location information of the target human, and the attribute information of the target object from the target image; and
capturing, based on the location information of the target human, the human image information of the target human corresponding to the location information of the target human from the target image.

### Claim 9

An electronic device, comprising:
at least one processor; and
a memory communicatively connected to the at least one processor, wherein
the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:
obtain attribute information of a target object and N pieces of candidate behavior-related information of a target human from a target image, wherein N is an integer greater than or equal to 1;
determine target behavior-related information based on comparison results between the N pieces of candidate behavior-related information and the attribute information of the target object; and
determine a behavior recognition result of the target human based on the target behavior-related information.

### Claim 10 (depends on 9)

The electronic device according to claim 9, wherein the instructions are executed by the at least one processor to enable the at least one processor to:
determine a comparison sequence of the N pieces of candidate behavior-related information based on confidence scores respectively corresponding to the N pieces of candidate behavior-related information;
select, based on the comparison sequence, a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information, for comparison with the attribute information of the target object; and
take the piece of candidate behavior to be compared-related information as the target behavior-related information, in a case that a comparison result is that the piece of candidate behavior to be compared-related information matches the attribute information of the target object.

### Claim 11 (depends on 9)

The electronic device according to claim 9, wherein the instructions are executed by the at least one processor to enable the at least one processor to:
determine, in a case that a category of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information matches a category of the target object, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object; and
determine, in a case that a difference between location information of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information and location information of the target object is within a preset range, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object.

### Claim 12 (depends on 9)

The electronic device according to claim 9, wherein the instructions are executed by the at least one processor to enable the at least one processor to:
take a category of a target behavior in the target behavior-related information as the behavior recognition result of the target human.

### Claim 13 (depends on 9)

The electronic device according to claim 9, wherein the instructions are executed by the at least one processor to enable the at least one processor to:
determine basic behavior-related information in the N pieces of candidate behavior-related information as the target behavior-related information, in a case that the comparison results indicate that all the N pieces of candidate behavior-related information do not match the attribute information of the target object.

### Claim 14 (depends on 9)

The electronic device according to claim 9, wherein the instructions are executed by the at least one processor to enable the at least one processor to:
obtain human image information of the target human from the target image; and
input the human image information into a classification model, to obtain the N pieces of candidate behavior-related information of the target human output by the classification model, wherein
a piece of candidate behavior-related information comprises at least one of a category and location information of a candidate behavior,

### Claim 15 (depends on 14)

The electronic device according to claim 14, wherein the instructions are executed by t at least one processor to enable the at least one processor to:
extract an image feature of the target image;
determine, based on the image feature, the target human, location information of the target human, and the attribute information of the target object from the target image; and
capture, based on the location information of the target human, the human image information of the target human corresponding to the location information of the target human from the target image.

### Claim 16

A non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are executed by a computer to enable the computer to:
obtain attribute information of a target object and N pieces of candidate behavior-related information of a target human from a target image, wherein N is an integer greater than or equal to 1;
determine target behavior-related information based on comparison results between the N pieces of candidate behavior-related information and the attribute information of the target object; and
determine a behavior recognition result of the target human based on the target behavior-related information.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein the computer instructions are executed by the computer to enable the computer to:
determine a comparison sequence of the N pieces of candidate behavior-related information based on confidence scores respectively corresponding to the N pieces of candidate behavior-related information;
select, based on the comparison sequence, a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information, for comparison with the attribute information of the target object; and
take the piece of candidate behavior to be compared-related information as the target behavior-related information, in a case that a comparison result is that the piece of candidate behavior to be compared-related information matches the attribute information of the target object.

### Claim 18 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein the computer instructions are executed by the computer to enable the computer to:
determine, in a case that a category of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information matches a category of the target object, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object; and
determine, in a case that a difference between location information of a candidate behavior to be compared comprised in a piece of candidate behavior to be compared-related information from the N pieces of candidate behavior-related information and location information of the target object is within a preset range, a comparison result that the piece of candidate behavior to be compared-related information matches the attribute information of the target object.

### Claim 19 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein the computer instructions are executed by the computer to enable the computer to:
take a category of a target behavior in the target behavior-related information as the behavior recognition result of the target human.

### Claim 20 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein the computer instructions are executed by the computer to enable the computer to:
determine basic behavior-related information in the N pieces of candidate behavior-related information as the target behavior-related information, in a case that the comparison results indicate that all the N pieces of candidate behavior-related information do not match the attribute information of the target object.
