Visual summarization of video for quick understanding by determining emotion objects for semantic segments of video

A method for rendering a summary of a video, comprising:

• determining, by a processor, one or more semantic segments of the video having metadata indicating a type of the video;
• determining, by the processor, one or more visual tags, one or more audio keyword tags, and one or more attribute tags associated with the one or more semantic segments of the video, wherein the one or more visual tags, the one or more audio keyword tags, and the one or more attribute tags relates to content of the one or more semantic segments of the video;
• validating, by the processor, the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags by comparing each of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags against one another for each of the determined one or more semantic segment of the video, wherein if a content identified by one of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags is not identified by comparison with another of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags, then the content identified by one of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags is discarded from further content analysis;
• generating, by the processor, one or more emotion objects by identifying patterns of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags that correspond or correlate with the one or more emotion object for each of the determined one or more semantic segment of the video for which the content identified by one of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags has not been discarded during validation, wherein each emotion object of the generated one or more emotion objects specifies an emotion or a mood that a corresponding one or more semantic segments of the video is expected to elicit from a viewer;
• generating, by the processor, a video summary interface on the display screen for presentation to the viewer, wherein the generated video summary interface comprises of a first window display region and a second window display region, wherein the horizontal extent of the first window display region of the video summary interface corresponds with the temporal length of a frame of the video;
• displaying, by the processor, the frame of the video within the second window display region of the video summary interface;
• displaying, by the processor, a timeline object within the first window display region of the video summary interface, wherein the timeline object corresponds to the temporal length of the frame of the video displayed in the second window display region of the video summary interface;
• displaying, by the processor, a first indicia associated with the generated one or more emotion objects within the first window display region of the video summary interface, wherein the first indicia is displayed within the first window display region of the video summary interface at a location corresponding with the temporal position appearance of the one or more emotion object in the video;
• displaying, by the processor, an audio object indicia associated with the determined one or more audio keyword tags within the first window display region of the video summary interface, wherein the audio object indicia is displayed within the first window display region of the video summary interface at a temporal location corresponding with the temporal rendering of the determined one or more audio keyword tags in the frame of the video;
• and
• displaying, by the processor, a key word object indicia associated with the determined one or more audio keyword tags within the first window display region of the video summary interface, wherein the key word object indicia is displayed within the first window display region of the video summary interface at the temporal location corresponding with the temporal rendering of the determined one or more audio keyword tags in the frame of the video.

A non-transitory computer-readable storage medium having executable code stored thereon to cause a machine to perform a method for rendering a summary of a video, comprising:

• determining, by a processor, one or more semantic segments of the video having metadata indicating a type of the video;
• determining, by the processor, one or more visual tags, one or more audio keyword tags, and one or more attribute tags associated with the one or more semantic segments of the video, wherein the one or more visual tags, the one or more audio keyword tags, and the one or more attribute tags relates to content of the one or more semantic segments of the video;
• validating, by the processor, the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags by comparing each of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags against one another for each of the determined one or more semantic segment of the video, wherein if a content identified by one of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags is not identified by comparison with another of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags, then the content identified by one of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags is discarded from further content analysis;
• generating, by the processor, one or more emotion objects by identifying patterns of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags that correspond or correlate with the one or more emotion object for each of the determined one or more semantic segment of the video for which the content identified by one of the determined one or more visual tags, the determined one or more audio keyword tags, and the determined one or more attribute tags has not been discarded during validation, wherein each emotion object of the generated one or more emotion objects specifies an emotion or a mood that a corresponding one or more semantic segments of the video is expected to elicit from a viewer;
• generating, by the processor, a video summary interface on the display screen for presentation to the viewer, wherein the generated video summary interface comprises of a first window display region and a second window display region, wherein the horizontal extent of the first window display region of the video summary interface corresponds with the temporal length of a frame of the video;
• displaying, by the processor, the frame of the video within the second window display region of the video summary interface;
• displaying, by the processor, a timeline object within the first window display region of the video summary interface, wherein the timeline object corresponds to the temporal length of the frame of the video displayed in the second window display region of the video summary interface;
• displaying, by the processor, a first indicia associated with the generated one or more emotion objects within the first window display region of the video summary interface, wherein the first indicia is displayed within the first window display region of the video summary interface at a location corresponding with the temporal position appearance of the one or more emotion object in the video;
• displaying, by the processor, an audio object indicia associated with the determined one or more audio keyword tags within the first window display region of the video summary interface, wherein the audio object indicia is displayed within the first window display region of the video summary interface at a temporal location corresponding with the temporal rendering of the determined one or more audio keyword tags in the frame of the video;
• and
• displaying, by the processor, a key word object indicia associated with the determined one or more audio keyword tags within the first window display region of the video summary interface, wherein the key word object indicia is displayed within the first window display region of the video summary interface at the temporal location corresponding with the temporal rendering of the determined one or more audio keyword tags in the frame of the video.