Contextual search on multimedia content

A non-transitory machine-readable medium comprising instructions stored therein, which when executed by a processor, causes the processor to perform operations comprising:

• receiving a query related to multimedia content, the query including one or more terms, being in addition to the multimedia content, and being provided by a user during consumption of the multimedia content by the user;
• identifying one or more terms in the query;
• automatically generating, for rewriting the query related to the multimedia content, one or more query rewrite candidates based on entities associated with the multimedia content and the one or more terms in the query, wherein the entities include values characterizing one or more objects represented in the multimedia content, and wherein generating the one or more query rewrite candidates comprises:
• scoring the one or more extracted entities based on one or more of:
• a time at which the extracted entities are annotated in the multimedia content or based on co-occurrences of n-grams in a query repository, the n-grams in the query repository each being a contiguous sequence of n items from a given sequence of text or speech, and the n-grams in the query repository including unigrams, bigrams, tri grams, and four-grams,
• ranking the extracted entities based on the scoring, and
• combining one or more of the terms of the query with one or more scored extracted entities to generate the one or more query rewrite candidates;
• providing the one or more query rewrite candidates to a search engine;
• scoring the one or more query rewrite candidates based on characteristics of respective result sets resulting from the providing;
• ranking the scored one or more query rewrite candidates based on their respective scores;
• rewriting the query related to the multimedia content based on a particular ranked query rewrite candidate;
• and
• providing for display, responsive to the query related to the multimedia content, a result set from the search engine based on the rewritten query.

A system comprising:

• a memory comprising instructions;
• and
• a processor configured to execute the instructions to:
• receive a query related to streaming multimedia content, wherein the query includes one or more terms and is provided by a user through voice input during consumption of the streaming multimedia content by the user;
• automatically generate, for rewriting the query related to the streaming multimedia content, one or more query rewrite candidates based on entities associated with the multimedia content and the one or more terms in the query, wherein the entities include values characterizing one or more objects represented in the multimedia content wherein in executing the instructions to generate the one or more query rewrite candidates, the processor is further to:
• score the one or more extracted entities based on one or more of:
• a time at which the extracted entities are annotated in the multimedia content or based on co-occurrences of n-grams in a query repository, the n-grams in the query repository each being a contiguous sequence of n items from a given sequence of text or speech, and the n-grams in the query repository including unigrams, bigrams, tri grams, and four-grams,
• rank the extracted entities based on the scoring, and
• combine one or more of the terms of the query with one or more scored extracted entities to generate the one or more query rewrite candidates;
• provide the one or more query rewrite candidates to a search engine;
• score the one or more query rewrite candidates based on characteristics of respective result sets resulting from the providing;
• rank the scored one or more query rewrite candidates based on their respective scores;
• rewrite the query related to the multimedia content based on a particular ranked query rewrite candidate;
• and
• provide, without interrupting consumption of the multimedia content, a result set from the search engine based on the rewritten query.