Back to results
US 12,743,891 B2Granted
End-to-end action recognition in intelligent video analysis and edge computing systems
AssigneeNVIDIA Corporation
InventorsSubhashree Radhakrishnan, Farzin Aghdasi
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques to perform action recognition. In at least one embodiment, action recognition is performed using one or more neural networks and hardware accelerators, in which the one or more neural networks are processed based on, for example, one or more quantization and pruning processes.
Claims20
A method, comprising:
- obtaining a plurality of frames of a video;
- determining one or more objects represented in the plurality of frames;
- causing a parallel processing unit to calculate a flow field representing movements of the one or more objects represented in one or more pixels among frames of the plurality of frames;
- obtaining one or more cropped movements by performing a first set of cropping operations on the flow field based at least in part on a set of bounding boxes corresponding to the objects;
- obtaining one or more cropped frames by performing a second set of cropping operations on the plurality of frames based at least in part on the set of bounding boxes;
- and
- causing a neural network to classify one or more actions performed by the one or more objects represented in the plurality of frames based at least in part on the one or more cropped movements and the one or more cropped frames.
4 more elements
A processor, comprising:
- one or more circuits to:
- identify one or more objects depicted in one or more frames from video data;
- calculate, using the one or more frames of the video data, one or more flow fields and one or more bounding boxes corresponding to the one or more objects;
- obtain one or more cropped movements by performing a first set of cropping operations on the one or more flow fields based, at least in part, on a set of bounding boxes corresponding to the objects;
- obtain one or more cropped frames by performing a second set of cropping operations on the one or more frames based at least in part on the set of bounding boxes;
- and
- cause one or more neural networks to classify one or more actions represented in the one or more frames based, at least in part, on the one or more cropped movements and the one or more cropped frames.
4 more elements
A computing device, comprising:
- a hardware accelerator;
- and
- memory comprising instructions executable by one or more processors of the computing device to at least:
- obtain a plurality of frames;
- cause the hardware accelerator to calculate a flow field representing movements of one or more objects represented by one or more pixels among frames of the plurality of frames;
- use a first neural network to determine the one or more objects represented in the plurality of frames;
- obtain one or more cropped movements by performing a first set of cropping operations on the flow field based at least in part on a set of bounding boxes corresponding to the objects;
- obtain one or more cropped frames by performing a second set of cropping operations on the plurality of frames based at least in part on the set of bounding boxes;
- and
- cause a second neural network to use the one or more cropped movements and the one or more cropped frames to classify one or more actions.
7 more elements
Dependencies are inferred from the claim text.