Video retrieval method and apparatus based on key frame detection
AssigneeTwelve Labs, Inc.
InventorsSeung Joon Lee, Sung Jun Kim, Raehyuk Jung, Haram Jo
Life of this patent
Estimating the term…
Abstract
In order to implement the foregoing object, an exemplary embodiment of the present disclosure discloses a video retrieval method performed by a computing device. The video retrieval method may include: generating one or more sub video data based on one or more video data; identifying, by a machine learning enabled key frame detecting module having one or more encoders, key frame information based on one or more encoded vectors generated from the one or more sub video data; and segmenting the one or more video data into one or more retrieval video data based on the identified key frame information.
Claims16
A video retrieval method performed by a computing device, the video retrieval method comprising:
- generating one or more sub video data based on one or more video data;
- generating one or more unit video data based on the one or more sub video data;
- generating, by a machine learning enabled key frame detecting module having one or more encoders, one or more key frame detection vectors for each of the one or more unit video data before detecting whether each of the one or more unit video data is a key frame;
- identifying key frame information based on the one or more key frame detection vectors generated from each unit video data of the one or more unit video data, wherein the key frame information indicates whether each unit video data is a key frame or not based on the respective key frame detection vector for the unit video data;
- and
- segmenting the one or more video data into one or more retrieval video data based on the identified key frame information.
A non-transitory computer readable storage medium storing a computer program, in which when the computer program is executed in one or more processors, the computer program causes the one or more processors to perform operations for performing a video retrieval method, the video retrieval method comprising:
- generating one or more sub video data based on one or more video data;
- generating one or more unit video data based on the one or more sub video data;
- generating, by a machine learning enabled key frame detecting module having one or more encoders, one or more key frame detection vectors for each of the one or more unit video data before detecting whether each of the one or more unit video data is a key frame;
- identifying key frame information based on the one or more key frame detection vectors generated from each unit video data of the one or more unit video data, wherein the key frame information indicates whether each unit video data is a key frame or not based on the respective key frame detection vector for the unit video data;
- and
- segmenting the one or more video data into one or more retrieval video data based on the identified key frame information.
A computing device performing a video retrieval method, the computing device comprising:
- a processor including at least one core;
- and
- a memory including program codes executable in the processor,
- wherein the processor:
- generates one or more sub video data based on one or more video data;
- generates one or more unit video data based on the one or more sub video data;
- generate, by a machine learning enabled key frame detecting module having one or more encoders, one or more key frame detection vectors for each of the one or more unit video data before detecting whether each of the one or more unit video data is a key frame;
- identifies key frame information based on the one or more key frame detection vectors generated from each unit video data of the one or more unit video data, wherein the key frame information indicates whether each unit video data is a key frame or not based on the respective key frame detection vector for the unit video data;
- and
- segments the one or more video data into one or more retrieval video data based on the identified key frame information.
Dependencies are inferred from the claim text.