# US 12,002,219 B2 · Identifying one or more events in content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,002,219 B2 - Document ID: US12002219B2 - Status: Granted - Kind: B2 - Application number: 17/896,682 - Filed: 2022-08-26 - Published: 2023-02-09 - Granted: 2024-06-04 - Earliest claimed: 2014-10-15 - Assignees: Comcast Cable Communications, LLC - Applicants: Comcast Cable Communications, LLC - Inventors: Erik Schwartz; Jan Neumann; Hans Sayyadi; Stefan Deichmann - Web page: https://patentgrep.com/patent/US12002219B2 ## CPC - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06F 16/48: Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually - G06F 16/745: {the internal structure of a single video sequence} - G06F 16/7867: {using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings} - G06V 20/42: {of sport video content} - G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes} - H04N 5/783: Adaptations for reproducing at a rate different from the recording rate - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} - H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments} - G06T 2207/10016: Video; Image sequence - G06T 2207/30221: Sports video; Sports image - G06V 20/44: {Event detection} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract According to some aspects, methods and systems may include receiving, by a computing device, metadata identifying an event occurring in a video program, and determining an expected motion of objects in the identified event. The methods and systems may further include analyzing motion energy in the video program to identify video frames in which the event occurs, and storing information identifying the video frames in which the event occurs. ## Claims ### Claim 1 A method comprising: receiving, by a computing device, metadata indicative of a time for a beginning of an event occurring in a video program and a time for an ending of the event; determining, based on the metadata and based on a plurality of video frames associated with the video program: a first video frame, from the plurality of video frames, comprising a first displayed time before the time for the beginning of the event; and a second video frame, from the plurality of video frames, comprising a second displayed time after the time for the ending of the event; selecting a portion of the video program, the portion comprising the first video frame, the second video frame, and video frames between the first video frame and the second video frame; determining an expected motion energy of objects associated with the event; determining, based on a comparison of motion energy of objects in the selected portion of the video program with the expected motion energy, one or more video frames, within the selected portion, in which the event occurs; and storing information indicating the one or more video frames in which the event occurs. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the first video frame and the second video frame is based on performing optical character recognition on the plurality of video frames associated with the video program. ### Claim 3 (depends on 1) The method of claim 1, wherein the first video frame comprises a first game clock showing the first displayed time, and wherein the second video frame comprises a second game clock showing the second displayed time. ### Claim 4 (depends on 1) The method of claim 1, wherein a first game clock showing a third displayed time corresponding to the time for the beginning of the event is not visible in the video program, and wherein a second game clock showing a fourth displayed time corresponding to the time for the ending of the event is not visible in the video program. ### Claim 5 (depends on 1) The method of claim 1, further comprising: dividing one or more frames of the portion of the video program into a plurality of sections; determining motion vectors of objects associated with one or more of the sections; and wherein the determining the one or more video frames in which the event occurs is performed based on the motion vectors of the objects associated with the one or more of the sections and based on a rule indicative of expected motion vectors of the objects associated with the event. ### Claim 6 (depends on 1) The method of claim 1, further comprising: determining a change in display view in the portion of the video program from a first display view to a second display view; and wherein the determining the one or more video frames in which the event occurs is performed based on the change in display view. ### Claim 7 (depends on 1) The method of claim 1, further comprising: receiving an indication of one or more environmental features in the one or more video frames in which the event occurs; determining, based on the indication of the one or more environmental features, a key frame of the one or more video frames in which the event occurs; and causing display of the key frame as a user-selectable thumbnail. ### Claim 8 (depends on 1) The method of claim 1, wherein the determining the first video frame is performed based on one or more first audio features associated with the plurality of video frames; and wherein the determining the second video frame is performed based on one or more second audio features associated with the plurality of video frames. ### Claim 9 An apparatus comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the apparatus to: receive metadata indicative of a time for a beginning of an event occurring in a video program and a time for an ending of the event; determine, based on the metadata and based on a plurality of video frames associated with the video program: a first video frame, from the plurality of video frames, comprising a first displayed time before the time for the beginning of the event; and a second video frame, from the plurality of video frames, comprising a second displayed time after the time for the ending of the event; select a portion of the video program, the portion comprising the first video frame, the second video frame, and video frames between the first video frame and the second video frame; determine an expected motion energy of objects associated with the event; determine, based on a comparison of motion energy of objects in the selected portion of the video program with the expected motion energy, one or more video frames, within the selected portion, in which the event occurs; and store information indicating the one or more video frames in which the event occurs. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein the instructions, when executed by the one or more processors, cause the apparatus to: determine the first video frame and the second video frame based on performing optical character recognition on the plurality of video frames associated with the video program. ### Claim 11 (depends on 9) The apparatus of claim 9, wherein the first video frame comprises a first game clock showing the first displayed time, and wherein the second video frame comprises a second game clock showing the second displayed time. ### Claim 12 (depends on 9) The apparatus of claim 9, wherein a first game clock showing a third displayed time corresponding to the time for the beginning of the event is not visible in the video program, and wherein a second game clock showing a fourth displayed time corresponding to the time for the ending of the event is not visible in the video program. ### Claim 13 (depends on 9) The apparatus of claim 9, wherein the instructions, when executed by the one or more processors, cause the apparatus to: divide one or more frames of the portion of the video program into a plurality of sections; determine motion vectors of objects associated with one or more of the sections; and determine the one or more video frames in which the event occurs based on the motion vectors of the objects associated with the one or more of the sections and based on a rule indicative of expected motion vectors of the objects associated with the event. ### Claim 14 (depends on 9) The apparatus of claim 9, wherein the instructions, when executed by the one or more processors, cause the apparatus to: determine a change in display view in the portion of the video program from a first display view to a second display view; and determine the one or more video frames in which the event occurs based on the change in display view. ### Claim 15 (depends on 9) The apparatus of claim 9, wherein the instructions, when executed by the one or more processors, cause the apparatus to: receive an indication of one or more environmental features in the one or more video frames in which the event occurs; determine, based on the indication of the one or more environmental features, a key frame of the one or more video frames in which the event occurs; and cause display of the key frame as a user-selectable thumbnail. ### Claim 16 A non-transitory computer-readable medium storing instructions that, when executed, cause: receiving, by a computing device, metadata indicative of a time for a beginning of an event occurring in a video program and a time for an ending of the event; determining, based on the metadata and based on a plurality of video frames associated with the video program: a first video frame, from the plurality of video frames, comprising a first displayed time before the time for the beginning of the event; and a second video frame, from the plurality of video frames, comprising a second displayed time after the time for the ending of the event; selecting a portion of the video program, the portion comprising the first video frame, the second video frame, and video frames between the first video frame and the second video frame; determining an expected motion energy of objects associated with the event; determining, based on a comparison of motion energy of objects in the selected portion of the video program with the expected motion energy, one or more video frames, within the selected portion, in which the event occurs; and storing information indicating the one or more video frames in which the event occurs. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the instructions, when executed, further cause: determining the first video frame and the second video frame based on performing optical character recognition on the plurality of video frames associated with the video program. ### Claim 18 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the first video frame comprises a first game clock showing the first displayed time, and wherein the second video frame comprises a second game clock showing the second displayed time. ### Claim 19 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein a first game clock showing a third displayed time corresponding to the time for the beginning of the event is not visible in the video program, and wherein a second game clock showing a fourth displayed time corresponding to the time for the ending of the event is not visible in the video program. ### Claim 20 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the instructions, when executed, further cause: dividing one or more frames of the portion of the video program into a plurality of sections; determining motion vectors of objects associated with one or more of the sections; and wherein the determining the one or more video frames in which the event occurs is performed based on the motion vectors of the objects associated with the one or more of the sections and based on a rule indicative of expected motion vectors of the objects associated with the event.