Inferring topics with entity linking and ontological data

A system for inferring topics from a multimodal file, the system comprising:

• an entity extraction component comprising an object detection component and a speech to text component, operative to extract a set of entities from a multimodal file comprising a video portion and an audio portion;
• an entity linking component operative to link the extracted set of entities to produce a set of linked entities;
• an information retrieval component operative to obtain reference information for the extracted set of entities;
• a graphing and analysis component operative to:
• generate a graph of the set of linked entities, the graph comprising nodes and edges;
• based at least on the nodes and edges of the graph, determine clusters in the graph;
• based at least on the clusters in the graph, identify topic candidates;
• and
• extract features from the clusters in the graph;
• a TopicID selection component operative to:
• rank the topic candidates within at least one cluster;
• and
• based at least on the ranking, select at least one TopicID from among the topic candidates to represent at least one cluster;
• and
• a video indexer operative to index the multimodal file with the at least one TopicID.

One or more computer storage devices having computer-executable instructions stored thereon for inferring topics from a multimodal file, which, on execution by a computer, cause the computer to perform operations comprising:

• receiving a multimodal file comprising a video portion and an audio portion;
• extracting a set of entities from the multimodal file, wherein extracting a set of entities from the multimodal file comprises:
• detecting objects in the video portion of the multimodal file with face recognition;
• detecting text in the audio portion of the multimodal file with a speech to text process;
• and
• disambiguating among a set of detected entity names;
• linking the set of entities to produce a set of linked entities;
• obtaining reference information for the set of entities;
• based at least on the reference information, generating a graph of the set of linked entities, the graph comprising nodes and edges;
• based at least on the nodes and edges of the graph, determining clusters in the graph;
• determining significant clusters and insignificant clusters in the determined clusters;
• based at least on the significant clusters in the graph, identifying topic candidates;
• extracting features from the significant clusters in the graph;
• based at least on the extracted features, mapping the topic candidates into a probability interval;
• based at least on the mapping, ranking the topic candidates within at least one significant cluster,
• based on the ranking, selecting at least one TopicID from among the topic candidates to represent the at least one significant cluster;
• and
• indexing the multimodal file with the at least one TopicID.