Back to results
US 12,111,866 B2Granted
Term weight generation method, apparatus, device and medium
AssigneeTencent Technology (Shenzhen) Company Limited
InventorJianhui Huang
Life of this patent
Estimating the term…
Abstract
A term weight determination method includes: obtaining a video and video-associated text, the video-associated text including at least one term; generating a halfway vector of the term by performing multimodal feature fusion on the features of the video, the video-associated text and the at least one term; and generating the weight of the at least one term based on the halfway vector of the at least one term.
Claims19
A term weight generation method performed by a computer device, comprising:
- obtaining a video and video-associated text, the video-associated text comprising at least one term, and the video-associated text being text information associated with the content of the video;
- generating a halfway vector of the at least one term by performing multimodal feature fusion on features of the video, the video-associated text and the at least one term by:
- extracting a video feature vector of the video;
- extracting a text feature vector of the video-associated text;
- segmenting the video-associated text to obtain a plurality of terms, the plurality of terms including the at least one term;
- extracting, using a deep neural network, a term feature vector of the at least one term;
- and
- fusing the video feature vector, the text feature vector and the term feature vector into the halfway vector of the at least one term;
- generating a weight of the at least one term based on dimension transformation of the halfway vector of the at least one term into a one-dimension parameter to enhance representation of the at least one term for content of the video and the video-associated text;
- calculating a similarity score between the at least one term and a search term in a video search request submitted by a terminal according to the weight of the at least one term;
- and
- returning the video to the requesting terminal when the similarity score is greater than a predefined threshold.
9 more elements
An electronic device, comprising:
- one or more processors;
- and
- memory storing one or more programs, the one or more programs comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
- obtaining a video and video-associated text, the video-associated text comprising at least one term, and the video-associated text being text information associated with the content of the video;
- generating a halfway vector of the at least one term by performing multimodal feature fusion on features of the video, the video-associated text and the at least one term by:
- extracting a video feature vector of the video;
- extracting a text feature vector of the video-associated text;
- segmenting the video-associated text to obtain a plurality of terms, the plurality of terms including the at least one term;
- extracting, using a deep neural network, a term feature vector of the at least one term;
- and
- fusing the video feature vector, the text feature vector and the term feature vector into the halfway vector of the at least one term;
- generating a weight of the at least one term based on dimension transformation of the halfway vector of the at least one term into a one-dimension parameter to enhance representation of the at least one term for content of the video and the video-associated text;
- calculating a similarity score between the at least one term and a search term in a video search request submitted by a terminal according to the weight of the at least one term;
- and
- returning the video to the requesting terminal when the similarity score is greater than a predefined threshold.
12 more elements
A non-transitory computer-readable storage medium, storing a computer program, the computer program, when executed by one or more processors of an electronic device, cause the one or more processors to perform operations comprising:
- obtaining a video and video-associated text, the video-associated text comprising at least one term, and the video-associated text being text information associated with the content of the video;
- generating a halfway vector of the at least one term by performing multimodal feature fusion on features of the video, the video-associated text and the at least one term by:
- extracting a video feature vector of the video;
- extracting a text feature vector of the video-associated text;
- segmenting the video-associated text to obtain a plurality of terms, the plurality of terms including the at least one term;
- extracting, using a deep neural network, a term feature vector of the at least one term;
- and
- fusing the video feature vector, the text feature vector and the term feature vector into the halfway vector of the at least one term;
- generating a weight of the at least one term based on dimension transformation of the halfway vector of the at least one term into a one-dimension parameter to enhance representation of the at least one term for content of the video and the video-associated text;
- calculating a similarity score between the at least one term and a search term in a video search request submitted by a terminal according to the weight of the at least one term;
- and
- returning the video to the requesting terminal when the similarity score is greater than a predefined threshold.
9 more elements
Dependencies are inferred from the claim text.