US 2022/0189210 A1 · Occlusion-Aware Prediction of Human Behavior

Abstract

An occlusion analysis system improves accuracy of behavior prediction models by generating occlusion parameters that may inform mathematical models to generate more accurate predictions. The occlusion analysis system trains and applies models for generating occlusion parameters, such as a manner in which a person is occluded, occlusion percentage, occlusion type. A behavior prediction system may input the occlusion parameters as well as other parameters relating to activity of the human into a second mathematical model for behavior prediction. The second machine learning model is a higher-level model trained to output a prediction that the human will exhibit a future behavior and a confidence level associated with the prediction. The confidence level is at least partially determined based on the occlusion parameters. The behavior prediction system may output the prediction and the confidence level to a control system that generates commands associated with a vehicle and other intelligent video analytics systems.

Claims20

Dependent claims

A method comprising:

receiving a video comprising a sequence of frames where at least a part of a human is visible in a given frame of the sequence of frames;
applying the sequence of frames to an occlusion model, wherein the occlusion model is a machine learning model trained to detect occlusion of the human in the video;
receiving, as output from the occlusion model, one or more occlusion parameters;
inputting the one or more occlusion parameters as well as other parameters relating to activity of the human into a second model that predicts that the human will exhibit a future behavior or recognizes a current behavior of the human based on the other parameters, wherein the second model outputs a confidence level associated with the prediction that is based on the one or more occlusion parameters;
and
outputting the prediction and the confidence level to a control system.

3 more elements

A non-transitory computer-readable storage medium comprising memory with executable computer instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to perform operations, the instructions comprising instructions to:

receive a video comprising a sequence of frames where at least a part of a human is visible in a given frame of the sequence of frames;
apply the sequence of frames to an occlusion model, wherein the occlusion model is a machine learning model trained to detect occlusion of the human in the video;
receive, as output from the occlusion model, one or more occlusion parameters;
input the one or more occlusion parameters as well as other parameters relating to activity of the human into a second model that predicts that the human will exhibit a future behavior or recognizes a current behavior of the human based on the other parameters, and to also output a confidence level associated with the prediction that is based on the one or more occlusion parameters;
and
output the prediction and the confidence level to a control system.

3 more elements

Dependencies are inferred from the claim text.

Occlusion-Aware Prediction of Human Behavior

Life of this patent

Occlusion-Aware Prediction of Human Behavior

Life of this patent

Abstract

Claims20