# US 12,217,506 B2 · Methods and systems for scoreboard text region detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,506 B2 - Document ID: US12217506B2 - Status: Granted - Kind: B2 - Application number: 18/196,310 - Filed: 2023-05-11 - Published: 2023-09-07 - Granted: 2025-02-04 - Earliest claimed: 2019-01-25 - Assignees: Gracenote, Inc. - Applicants: Gracenote, Inc. - Inventors: Jeffrey Scott; Markus Kurt Peter Cremer; Nishit Umesh Parekh; Dewey Ho Lee - Web page: https://patentgrep.com/patent/US12217506B2 ## CPC - G06V 20/42: {of sport video content} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - G06T 7/13: Edge detection - G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images - H04N 21/2187: Live feed - H04N 21/26603: {for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques} - H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window} - H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A computing system engages in digital image processing of received video frames to generate sport data that indicates a score and/or a time associated with a sport event. The digital image processing includes: (i) identifying a first frame region of the video frames based on the first frame region depicting a scoreboard; (ii) executing a first procedure that analyzes the identified first frame region to detect, within the identified first frame region, second frame region(s) based on the second frame region(s) depicting text of the scoreboard; (iii) in response to detecting the second frame region(s), executing a second procedure to recognize the text in at least one of the second frame region(s); and (iv) based at least on the recognizing of the text, generating the sport data. In response to completing the digital image processing, the computing system then carries out an action based on the generated sport data. ## Claims ### Claim 1 A method comprising: receiving, by a computing system, a time sequence of digital video frames that depict media content, wherein the computing system comprises a processing unit; engaging, by the computing system, in digital image processing of the received time sequence of digital video frames to automatically generate data, wherein the digital image processing comprises: (i) identifying a first frame region of the digital video frames based on the first frame region depicting a scoreboard, (ii) executing a first machine-driven pattern recognition procedure that includes analyzing the identified first frame region to detect, within the identified first frame region, one or more second frame regions based on the second frame regions depicting text of the scoreboard, (iii) in response to detecting the second frame regions, executing a second machine-driven pattern recognition procedure to recognize the text in at least one of the one or more second frame regions, and (iv) based at least on the recognizing of the text, generating the data; and in response to completing the digital image processing, carrying out, by the computing system, an action based on the generated data. ### Claim 2 (depends on 1) The method of claim 1, wherein receiving the time sequence of digital video frames comprises receiving a real-time broadcast feed that includes the digital video frames. ### Claim 3 (depends on 2) The method of claim 2, wherein engaging in the digital image processing comprises engaging in the digital image processing while receiving the real-time broadcast feed. ### Claim 4 (depends on 1) The method of claim 1, further comprising: receiving, by the computing system, electronic program guide (EPG) data that indicates event types respectively associated with a plurality of broadcast feeds, wherein a particular broadcast feed of the plurality includes the digital video frames; and based on the received EPG data, making a determination, by the computing system, that the particular broadcast feed is associated with an event, wherein engaging in the digital image processing comprises triggering the digital image processing in response to making the determination. ### Claim 5 (depends on 4) The method of claim 4, wherein the EPG data also indicates times respectively associated with scheduling of the plurality of broadcast feeds, the method further comprising: based on the received EPG data, determining, by the computing system, a particular time associated with scheduling of the particular broadcast feed that includes the time sequence of digital video frames, and wherein engaging in the digital image processing comprises, based on the determining of the particular time, triggering the digital image processing at the determined particular time. ### Claim 6 (depends on 1) The method of claim 1, further comprising: receiving, by the computing system, a request to trigger the digital image processing, wherein engaging in the digital image processing comprises triggering the digital image processing in response the received request. ### Claim 7 (depends on 1) The method of claim 1, wherein generating the data comprises generating data that indicates one or more of the following: (i) a first team name, (ii) a second team name, (iii) a first score associated with the first team name, and (iv) a second score associated with the second team name. ### Claim 8 (depends on 1) The method of claim 1, further comprising: receiving, by the computing system, information indicating the first frame region that depicts the scoreboard, wherein identifying the first frame region is based on the received information indicating the first frame region. ### Claim 9 (depends on 1) The method of claim 1, wherein identifying the first frame region comprises executing a third machine-driven pattern recognition procedure that analyzes one or more of the digital video frames to identify the first frame region depicting the scoreboard. ### Claim 10 (depends on 9) The method of claim 9, wherein the third machine-driven pattern recognition procedure analyzes at least a particular one of the digital video frames to identify the first frame region depicting the scoreboard, and wherein the first machine-driven pattern recognition procedure analyzes the first frame region within the particular video frame to detect the one or more second frame regions that depict text of the scoreboard. ### Claim 11 (depends on 9) The method of claim 9, wherein the first frame region appears in at least a first and a second of the digital video frames, wherein the third machine-driven pattern recognition procedure analyzes at least the first digital video frame to identify the first frame region depicting the scoreboard, and wherein the first machine-driven pattern recognition procedure analyzes the first frame region within the second digital video frame to detect the one or more second frame regions that depict text of the scoreboard. ### Claim 12 (depends on 1) The method of claim 1, wherein the digital image processing further comprises recording one or more of (i) a location of the identified first frame region within the digital video frames, (ii) an area within the digital video frames that the identified first frame region encompasses, (iii) a location of the detected one or more second frame regions within the digital video frames, and (iv) an area within the digital video frames that the detected one or more second frame regions encompass. ### Claim 13 (depends on 1) The method of claim 1, wherein generating the data comprises generating data that indicates a game time. ### Claim 14 (depends on 1) The method of claim 1, wherein carrying out the action comprises one or more of the following: (i) storing the generated data in a data storage device, (ii) causing presentation of the generated data, (iii) outputting a report related to the generated data, (iv) updating information in a user-account based on the generated data, and (v) causing presentation of supplemental content based on the generated data. ### Claim 15 A computing system comprising: a processing unit; non-transitory data storage; and program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations for automatically detecting, in a sequence of video frames, a video frame region that depicts a scoreboard, the operations comprising: receiving a time sequence of digital video frames that depict media content, wherein the computing system comprises a processing unit; engaging in digital image processing of the received digital video frames to automatically generate data, wherein the digital image processing comprises: (i) identifying a first frame region of the digital video frames based on the first frame region depicting a scoreboard, (ii) executing a first machine-driven pattern recognition procedure that includes analyzing the identified first frame region to detect, within the identified first frame region, one or more second frame regions based on the second frame regions depicting text of the scoreboard, (iii) in response to detecting the second frame regions, executing a second machine-driven pattern recognition procedure to recognize the text in at least one of the one or more second frame regions, and (iv) based at least on the recognizing of the text, generating the data; and in response to completing the digital image processing, carrying out an action based on the generated data. ### Claim 16 (depends on 15) The computing system of claim 15, wherein receiving the time sequence of digital video frames comprises receiving a real-time broadcast feed that includes the digital video frames, and wherein engaging in the digital image processing comprises engaging in the digital image processing while receiving the real-time broadcast feed. ### Claim 17 (depends on 15) The computing system of claim 15, wherein the operations further comprise: receiving, by the computing system, information indicating the first frame region that depicts the scoreboard, wherein identifying the first frame region is based on the received information indicating the first frame region. ### Claim 18 (depends on 15) The computing system of claim 15, wherein identifying the first frame region comprises executing a third machine-driven pattern recognition procedure that analyzes one or more of the digital video frames to identify the first frame region depicting the scoreboard. ### Claim 19 (depends on 15) The computing system of claim 15, wherein carrying out the action comprises one or more of the following: (i) storing the generated data in a data storage device, (ii) causing presentation of the generated data, (iii) outputting a report related to the generated data, (iv) updating information in a user-account based on the generated data, and (v) causing presentation of supplemental content based on the generated data. ### Claim 20 A non-transitory computer readable medium having stored thereon instructions executable by a processing unit to cause a computing system to perform operations comprising: receiving, by a computing system, a time sequence of digital video frames that depict media content, wherein the computing system comprises a processing unit; engaging, by the computing system, in digital image processing of the received digital video frames to automatically generate data, wherein the digital image processing comprises: (i) identifying a first frame region of the digital video frames based on the first frame region depicting a scoreboard, (ii) executing a first machine-driven pattern recognition procedure that includes analyzing the identified first frame region to detect, within the identified first frame region, one or more second frame regions based on the second frame regions depicting text of the scoreboard, (iii) in response to detecting the second frame regions, executing a second machine-driven pattern recognition procedure to recognize the text in at least one of the one or more second frame regions, and (iv) based at least on the recognizing of the text, generating the data; and in response to completing the digital image processing, carrying out, by the computing system, an action based on the generated data.