Video retrieval method and apparatus using vectorizing segmented videos

A non-transitory computer-readable storage medium storing a computer program, in which when the computer program is executed in one or more processors, the computer program causes the one or more processors to perform operations for performing a video retrieval method, the video retrieval method comprising:

• identifying, by a machine learning enabled key frame detecting module having one or more encoders, key frame information for one or more video data based on one or more encoded vectors respectively generated from one or more unit video data included in the one or more video data;
• segmenting the one or more video data into one or more target retrieval video data based on the identified key frame information;
• and
• generating, by a machine learning enabled retrieval vector generating module having one or more retrieval video encoding modules, one or more retrieval video vectors respectively representing the one or more target retrieval video data, wherein the generating further comprising:
• for each target retrieval data, generating, by the one or more retrieval video encoding modules, two or more different domain-based retrieval video embedding tokens corresponding to the target retrieval video data, wherein the two or more different domain-based retrieval video embedding tokens are generated based on two or more data domains included in the target retrieval video data, and
• generating the one or more retrieval video vectors from the two or more different domain-based retrieval video embedding tokens of the one or more target retrieval video data, comprising:
• generating a time embedding token for a position,
• combining the two or more different domain-based retrieval video embedding tokens and the time embedding token to generate a combined token, and
• generating a retrieval video vector in the one or more retrieval video vectors by applying a retrieval token encoder to the combined token.

A computing device performing a video retrieval method, the computing device comprising:

• a processor including at least one core;
• and
• a memory including program codes executable in the processor,
• wherein the processor:
• identifies, by a machine learning enabled key frame detecting module having one or more encoders, key frame information for one or more video data based on one or more encoded vectors respectively generated from one or more unit video data included in the one or more video data,
• segments the one or more video data into one or more target retrieval video data based on the identified key frame information, and
• generates by a machine learning enabled retrieval vector generating module having one or more retrieval video encoding modules, one or more retrieval video vectors respectively representing the one or more target retrieval video data, wherein to generate the one or more retrieval video vectors, the processor:
• for each target retrieval data, generate, by the one or more retrieval video encoding modules, two or more different domain-based retrieval video embedding tokens corresponding to the target retrieval video data, wherein the two or more different domain-based retrieval video embedding tokens are generated based on two or more data domains included in the target retrieval video data, and
• generate the one or more retrieval vectors from the two or more different domain-based retrieval video embedding tokens of the one or more target retrieval video data, wherein the processor further:
• generates a time embedding token for a position,
• combines the two or more different domain-based retrieval video embedding tokens and the time embedding token to generate a combined token, and
• generates a retrieval video vector in the one or more retrieval video vectors by applying a retrieval token encoder to the combined token.