# US 9,438,850 B2 · Determining importance of scenes based upon closed captioning data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,438,850 B2 - Document ID: US9438850B2 - Status: Granted - Kind: B2 - Application number: 14/938,554 - Filed: 2015-11-11 - Published: 2016-03-03 - Granted: 2016-09-06 - Earliest claimed: 2013-03-08 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Jeromey Russell Goetz - Web page: https://patentgrep.com/patent/US9438850B2 ## CPC - H04N 7/0882: {for the transmission of character code signals, e.g. for teletext (circuits for the digital non-picture data signal H04N7/035)} - G06F 16/7844: {using original textual content or text extracted from visual content or transcript of audio data} - H04H 60/377: {Scene} - H04N 21/233: Processing of audio elementary streams {(monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)} - H04N 21/4884: {for displaying subtitles} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/8549: Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N) ## Abstract Disclosed are various embodiments for identifying importance of scenes or moments in video content relative to one another. Closed captioning data is extracted from a video content feature. A textual analysis of the closed captioning data is performed. The importance level of scenes can be ranked with respect to one another. ## Claims ### Claim 1 A non-transitory computer-readable medium embodying a program executable in at least one computing device, the program, when executed, causing the at least one computing device to at least: obtain a video content feature, the video content feature associated with closed captioning data; perform an analysis of text in the closed captioning data in a plurality of scenes of the video content feature; and determine a respective importance of the plurality of scenes based at least in part upon the analysis of the text in the closed captioning data. ### Claim 2 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the program further causes the at least one computing device to at least rank the plurality of scenes of the video content feature according to the respective importance of the plurality of scenes. ### Claim 3 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the analysis of the text further comprises an analysis of spoken dialog in the closed captioning data. ### Claim 4 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the analysis of the text further comprises an analysis of non-dialog content in the closed captioning data, the analysis of the non-dialog content based at least in part upon whether the non-dialog content corresponds to narration or non-spoken sounds. ### Claim 5 A system, comprising: at least one computing device; and a video content analysis application executable in the at least one computing device, the video content analysis application, when executed, causing the at least one computing device to at least: extract closed captioning data from a video content feature; perform an analysis of text from the closed captioning data; and determine a respective importance of a plurality of scenes based at least in part upon the analysis of the text in the closed captioning data. ### Claim 6 (depends on 5) The system of claim 5, wherein the video content analysis application further causes the at least one computing device to at least rank a plurality of scenes of the video content feature based at least in part upon the analysis of text from the closed captioning data. ### Claim 7 (depends on 5) The system of claim 5, wherein the video content analysis application further causes the at least one computing device to at least identify the plurality of scenes in the video content feature by performing scene change detection within the video content feature. ### Claim 8 (depends on 5) The system of claim 5, wherein the respective importance of the plurality of scenes is determined based at least in part upon an amount of dialog occurring within individual ones of the plurality of scenes. ### Claim 9 (depends on 5) The system of claim 5, wherein the video content analysis application performs the analysis of text from the closed captioning data by: identifying a quotation corresponding to a portion of dialog from the video content feature as being associated with a scene in the video content feature; identifying the quotation within the closed captioning data; and ranking the scene corresponding to the quotation. ### Claim 10 (depends on 9) The system of claim 9, wherein the quotation corresponding to the portion of dialog from the video content feature is identified by determining whether the quotation corresponds to at least one of a plurality of user identified quotations associated with the video content feature, the plurality of user identified quotations being identified in a network site in communication with the at least one computing device. ### Claim 11 (depends on 5) The system of claim 5, wherein the analysis of text from the closed captioning data is performed via a sentiment analysis on the closed captioning data. ### Claim 12 (depends on 5) The system of claim 5, wherein the analysis of text from the closed captioning data is performed by: obtaining a synopsis associated with the video content feature from a network site in communication with the at least one computing device; identifying content from the synopsis in the closed captioning data; and ranking a scene corresponding to the content from the synopsis that is identified in the closed captioning data. ### Claim 13 (depends on 5) The system of claim 5, wherein the analysis of text from the closed captioning data is performed by: ranking a plurality of performers appearing within the video content feature according to prominence; and identifying the respective importance of a respective scene according to a ranking of a performer from the plurality of performers appearing within the respective scene. ### Claim 14 (depends on 13) The system of claim 13, wherein the video content analysis application further causes the at least one computing device to at least identify whether the performer appears within the respective scene by identifying a name of a character of the performer within the closed captioning data. ### Claim 15 (depends on 13) The system of claim 13, wherein ranking the plurality of performers appearing within the video content feature according to prominence is performed by at least one of: ranking the plurality of performers based at least in part upon an amount of dialog in the closed captioning data that is attributable to individual ones of the plurality of performers or ranking the plurality of performers based at least in part upon data obtained from a network site containing information about the plurality of performers appearing within video content feature. ### Claim 16 A method, comprising: obtaining, by at least one computing device, a video content feature associated with closed captioning data; extracting, by the at least one computing device, the closed captioning data from the video content feature; and determining, by the at least one computing device, a respective importance of a plurality of scenes based at least in part upon an analysis of text in the closed captioning data. ### Claim 17 (depends on 16) The method of claim 16, further comprising ranking a plurality of scenes of the video content feature based at least in part upon the analysis of text from the closed captioning data. ### Claim 18 (depends on 16) The method of claim 16, wherein the respective importance of the plurality of scenes is determined based at least in part upon an amount of dialog occurring within individual ones of the plurality of scenes. ### Claim 19 (depends on 16) The method of claim 16, wherein determining the respective importance of the plurality of scenes further comprises: ranking a plurality of performers appearing within the video content feature according to prominence; and identifying the respective importance of a respective scene according to a ranking of a performer from the plurality of performers appearing within the respective scene. ### Claim 20 (depends on 19) The method of claim 19, wherein ranking the plurality of performers appearing within the video content feature according to prominence is performed by at least one of: ranking the plurality of performers based at least in part upon an amount of dialog in the closed captioning data that is attributable to individual ones of the plurality of performers or ranking the plurality of performers based at least in part upon data obtained from a network site containing information about the plurality of performers appearing within video content feature.