Back to results
US 2025/0148768 A1Published application
Open vocabulary action detection
AssigneeNEC Laboratories America, Inc.
InventorsKai Li, Deep Patel, Renqiang Min, Wentao Bao
Life of this patent
Estimating the term…
Abstract
Methods and systems for action detection include encoding a text feature of an input textual description of an action using a visual language model (VLM). A video feature of an input video is encoded using the VLM. The action in the video is recognized, based on the text feature and the video feature, to localize the action within the video. A person performing the action is located within the video using the VLM.
Claims20
A computer-implemented method for action detection, comprising:
- encoding a text feature of an input textual description of an action using a visual language model (VLM);
- encoding a video feature of an input video using the VLM;
- recognizing the action in the video, based on the text feature and the video feature, to localize the action within the video;
- and
- locating a person performing the action within the video using the VLM.
2 more elements
A system for action detection, comprising:
- a hardware processor;
- and
- a memory that stores a computer program which, when executed by the hardware processor, causes the hardware processor to:
- encode a text feature of an input textual description of an action using a visual language model (VLM);
- encode a video feature of an input video using the VLM;
- recognize the action in the video, based on the text feature and the video feature, to localize the action within the video;
- and
- locate a person performing the action within the video using the VLM.
5 more elements
Dependencies are inferred from the claim text.