US 2009/0022472 A1 · Method and Apparatus for Video Digest Generation

Abstract

Automated video digest system and method. The system decomposes video and audio portions of video media into low-level descriptors, producing a large number of such descriptors for each video image and scene. These descriptors are optimized to be both compatible with rapid automated descriptor creation, and also to have some positive or negative statistical correlation with the viewing preferences of typical human viewers. By comparing the number of descriptors correlated with positive human viewing preferences, versus the number of descriptors correlated with negative human viewing preferences, the automated system may analyze an unknown video program and make educated guesses as to which portions of the program are likely to be most interesting to the viewer on playback. Less interesting portions may be skipped or fast-forwarded though. The method may also be optimized to individual viewers. Such methods are useful for controlling the playback of digital video recorders and other systems.

Claims58

Dependent claims

A digital video recorder system for storage and/or playback of media content, comprising:

at least one video processor capable of analyzing input media content and automatically generating descriptors that are associated with the image and/or audio contents of various time portions of the input media content;
at least one sensor to detect and record user viewing preferences while the input media content is being played;
persistent memory capable of storing a persistent database that correlates the descriptors with the recorded user viewing preferences;
at least one control processor, and associated control software;
in which the control processor, under the control of the control software, controls the storage and/or playback of various time portions of a previously unanalyzed input media content (unknown media content) by:
a) sending the unknown media content to the video processor, and directing the video processor to generate descriptors of the video and or audio portion of various time portions of the unknown media content (unknown video descriptors);
b) using the unknown video descriptors as an index to the persistent database, and retrieving the user viewing preferences associated with the unknown video descriptors;
c) constructing an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function;
and using this estimated user preference to control the storage and/or playback of various time portions of the unknown media content.

6 more elements

A digital video recorder device for storage and playback of media content, comprising:

at least one video processor capable of analyzing input media content and automatically generating descriptors that are associated with the image and audio contents of various time portions of the input media content;
at least one sensor to detect and record user viewing preferences while the input media content is being played;
persistent memory capable of storing a persistent database that correlates the descriptors with the recorded user viewing preferences;
at least one control processor and associated operational software;
in which the control processor, under the control of the control software, controls the storage and playback of various time portions of a previously unanalyzed input media content (unknown media content), the processor including a descriptor generator configured to receive the unknown media content and to generate descriptors of the video and or audio portion of various time portions of the unknown media content (unknown video descriptors);
an index unit configured to utilize the unknown video descriptors as an index to the persistent database to retrieve user viewing preferences associated with the unknown video descriptors;
and a preference constructor configured to construct an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function;
wherein the processor is further configured to utilize this estimated user preference to control the storage and/or playback of various time portions of the unknown media content.

5 more elements

A method of controlling the storage and/or playback of media content, comprising:

inputting a plurality of different media contents, and automatically generating a plurality of descriptors that are associated with the image and/or audio contents of various time portions of the media content;
and
detecting and recording the user viewing preferences associated with these various time portions of the different media contents;
correlating the descriptors with the user video viewing preferences in a persistent database so that the descriptors may be used as an index to the persistent database to retrieve user video viewing preferences;
and using this database to control the storage and/or playback of a previously unrated (unknown) media content by:
automatically generating a plurality of descriptors that are associated with the unknown media content (unknown content descriptors);
using the unknown content descriptors as an index into the database;
retrieving the user viewing preferences;
constructing an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function;
and using this estimated user preference to control the storage and/or playback of the unknown media content.

7 more elements

A method of controlling the storage and/or playback of media content, said method comprising:

inputting a plurality of different media contents, and automatically generating a plurality of descriptors that are associated with the image and/or audio contents of various time portions of the media content;
said descriptors being selected from population of descriptors known to have a positive or negative statistical correlation with user preferences;
said descriptors being the outputs of mathematical functions selected from the group consisting of three dimensional blocks of pixels, three dimensional edges, spatio-temporal regions with consistent motion, a priori objects, and collections of simple features;
or
said descriptors being the outputs of mathematical functions selected from the group consisting of functions that analyze sound frequency range, sound energy versus frequency range, voice frequency range, music frequency range, word recognition, and language recognition;
and
detecting and recording the user viewing preferences associated with these various time portions of the different media contents;
correlating the descriptors with the user video viewing preferences in a persistent database so that the descriptors may be used as an index to the persistent database to retrieve user video viewing preferences;
and using this database to control the storage and/or playback of a previously unrated (unknown) media content by:
automatically generating a plurality of descriptors that are associated with the unknown media content (unknown content descriptors);
using the unknown content descriptors as an index into the database;
retrieving the user viewing preferences;
constructing an estimated user preference of the various time portions of the unknown media content by using the retrieved user viewing preferences and a suitable weighing function;
and using this estimated user preference to control the storage and/or playback of the unknown media content.

11 more elements

Dependencies are inferred from the claim text.

Method and Apparatus for Video Digest Generation

Life of this patent

Method and Apparatus for Video Digest Generation

Life of this patent

Abstract

Claims58