Systems and methods for using latent variable modeling for multi-modal video indexing

A non-transitory computer-readable medium embodying a set of computer-executable instructions, which, when executed in connection with a computerized system comprising a processing unit and a memory, cause the computerized system to perform a method comprising:

• a. using the processing unit to generate a multi-modal language model for co-occurrence of spoken words in the plurality of videos and an external text associated with the plurality of videos;
• b. selecting at least a portion of a first video;
• c. extracting a plurality of spoken words from the selected portion of the first video;
• d. obtaining a first external text associated with the selected portion of the first video, wherein the obtained first external text is separate and distinct from a representation of the extracted plurality of spoken words;
• and
• e. using the processing unit and the generated multi-modal language model to rank the extracted plurality of spoken words based on probability of occurrence conditioned on the obtained first external text.

A computerized system comprising a processing unit and a memory storing a set of instructions, the set of instructions comprising instructions for:

• a. using the processing unit to generate a multi-modal language model for co-occurrence of spoken words in the plurality of videos and an external text associated with the plurality of videos;
• b. selecting at least a portion of a first video;
• c. extracting a plurality of spoken words from the selected portion of the first video;
• d. obtaining a first external text associated with the selected portion of the first video, wherein the obtained first external text is separate and distinct from a representation of the extracted plurality of spoken words;
• and
• e. using the processing unit and the generated multi-modal language model to rank the extracted plurality of spoken words based on probability of occurrence conditioned on the obtained first external text.