Back to results
US 2024/0193208 A1Published application
VideoChat
AssigneeMicrosoft Technology Licensing, LLC
InventorsNan Duan, Lei Ji, Ming Zhou
Life of this patent
Estimating the term…
Abstract
The present disclosure provides a technical solution of multi-modal chatting, which may provide response to user query by using multi-modal response in the interaction between chatbot and human beings, so that the expressing ways and the expressed content by the chatbot could be richer by using such response in a multi-modal way.
Claims20
A computing device comprising:
- a hardware processor;
- a memory, the memory storing instructions, which when executed, cause the hardware processor to perform operations comprising:
- acquiring, from a conversational artificial intelligence session with a user, a user query;
- performing a similarity matching calculation between the user query and each of a plurality of components of a first video, the components comprising audio, image, and subtitle data corresponding to the first video to create a plurality of first similarity scores, each of the plurality of first similarity scores describing a similarity between the user query and a single one of the plurality of components of the first video;
- combining the plurality of first similarity scores to generate a combined similarity score;
- selecting a video from a plurality of videos based upon the combined similarity score of the first video and a plurality of other combined similarity scores of other videos in the plurality of videos;
- generating a query response including the selected video;
- generating summary information of the selected video;
- and
- causing the query response to be output on a display device of the user as part of a response to the user query in the conversational artificial intelligence session, the output including the selected video and including the summary information of the selected video.
7 more elements
A method comprising:
- acquiring, from a conversational artificial intelligence session with a user, a user query;
- performing a similarity matching calculation between the user query and each of a plurality of components of a first video, the components comprising audio, image, and subtitle data corresponding to the first video to create a plurality of first similarity scores, each of the plurality of first similarity scores describing a similarity between the user query and a single one of the plurality of components of the first video;
- combining the plurality of first similarity scores to generate a combined similarity score;
- selecting a video from a plurality of videos based upon the combined similarity score of the first video and a plurality of other combined similarity scores of other videos in the plurality of videos;
- generating a query response including the selected video;
- generating summary information of the selected video;
- and
- causing the query response to be output on a display device of the user as part of a response to the user query in the conversational artificial intelligence session, the output including the selected video and including the summary information of the selected video.
5 more elements
A machine-readable medium, storing instructions, which when executed by a machine, cause the machine to perform operations comprising:
- acquiring, from a conversational artificial intelligence session with a user, a user query;
- performing a similarity matching calculation between the user query and each of a plurality of components of a first video, the components comprising audio, image, and subtitle data corresponding to the first video to create a plurality of first similarity scores, each of the plurality of first similarity scores describing a similarity between the user query and a single one of the plurality of components of the first video;
- combining the plurality of first similarity scores to generate a combined similarity score;
- selecting a video from a plurality of videos based upon the combined similarity score of the first video and a plurality of other combined similarity scores of other videos in the plurality of videos;
- generating a query response including the selected video;
- generating summary information of the selected video;
- and
- causing the query response to be output on a display device of the user as part of a response to the user query in the conversational artificial intelligence session, the output including the selected video and including the summary information of the selected video.
5 more elements
Dependencies are inferred from the claim text.