# US 11,443,512 B1 · Systems and methods for querying video information of electronic games > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,512 B1 - Document ID: US11443512B1 - Status: Granted - Kind: B1 - Application number: 17/246,050 - Filed: 2021-04-30 - Granted: 2022-09-13 - Earliest claimed: 2021-04-30 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Yash Vesikar; Varun Varshney; Donald Frank Brinkman - Web page: https://patentgrep.com/patent/US11443512B1 ## CPC - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06F 16/7834: {using audio features} - G06F 16/7837: {using objects detected or recognised in the video content} - G06F 16/7867: {using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06N 20/00: Machine learning - G06V 20/20: in augmented reality scenes - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G06V 20/44: {Event detection} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method of presenting video information to a user includes, at a server computer, obtaining video information including a plurality of frame from a real-time video stream of an electronic game and obtaining ML model data. The method includes determining a presence of at least one event in the video information using a ML model and the ML model data to identify the at least one event. Upon receiving a query from a user for the presence of the at least on event in the video information, the method includes denoting a presence of the at least one event in the video information associated with at least one frame of the plurality of frames of the video information and presenting to the user a visual representation of the at least one frame of the plurality of frames of the video information associated with the at least one event. ## Claims ### Claim 1 A method of presenting video information to a user, the method comprising: at a server computer: obtaining video information including a plurality of frames from an electronic game; obtaining machine learning (ML) model data including an application module including game characteristics associated with the electronic game; determining a presence of at least one event in the video information using a ML model and the ML model data to identify the at least one event and at least one game characteristic present during the at least one event; receiving a query from a user for the presence of the at least one event and the game characteristic present during the at least one event in the video information; denoting a presence of the at least one event in the video information associated with at least one frame of the plurality of frames of the video information; and presenting to the user a visual representation of the at least one frame of the plurality of frames of the video information associated with the at least one event. ### Claim 2 (depends on 1) The method of claim 1 further comprising amending metadata of the video information after determining the presence of the at least one event. ### Claim 3 (depends on 1) The method of claim 1, wherein presenting to the user a visual representation includes marking on a timeline of the video information the temporal location of the at least one frame. ### Claim 4 (depends on 1) The method of claim 1 further wherein the game characteristic includes a sound. ### Claim 5 (depends on 1) The method of claim 1, wherein the query includes a plurality of parameters. ### Claim 6 (depends on 1) The method of claim 1, wherein the ML model is an unsupervised model. ### Claim 7 (depends on 1) The method of claim 1, wherein the ML model is trained by video information from other users. ### Claim 8 (depends on 1) The method of claim 1, wherein the ML model is trained by social media metric of published video information. ### Claim 9 (depends on 1) The method of claim 1, wherein the video information is obtained from a client device that renders the video information. ### Claim 10 (depends on 1) The method of claim 1, further comprising presenting to the user a visual representation of at least three events automatically identified by the server computer. ### Claim 11 (depends on 1) The method of claim 1, wherein the at least one event includes a presence of an object in the video information. ### Claim 12 (depends on 1) The method of claim 1, wherein the at least one event occurs over at least two frames in the video information. ### Claim 13 (depends on 1) The method of claim 1, wherein the visual representation of the at least one frame of the plurality of frames of the video information associated with the at least one event includes an amplitude indicator. ### Claim 14 A method of presenting video information to a user, the method comprising: at a client computer: obtaining video information including a plurality of frames from an electronic game; obtaining ML model data from a remote server; determining a presence of at least one event in the video information using the ML model data; receiving a query for the presence of the at least one event in the video information; denoting a presence of the at least one event in the video information associated with at least one frame of the plurality of frames of the video information; and presenting to the user a visual representation of the at least one frame of the plurality of frames of the video information associated with the at least one event. ### Claim 15 (depends on 14) The method of claim 14 further comprising rendering the video information. ### Claim 16 (depends on 15) The method of claim 15, further comprising transmitting to the remote server information regarding the at least one event in the video information. ### Claim 17 (depends on 14) The method of claim 14, wherein determining a presence of at least one event includes indexing the plurality of frames and determining a plurality of events. ### Claim 18 (depends on 17) The method of claim 17, wherein denoting the presence of the at least one event includes associating a timestamp list with the video information wherein each event of the plurality of events is denoted with a timestamp in the timestamp list. ### Claim 19 (depends on 18) The method of claim 18, wherein the timestamp list is a separate data file from the video information. ### Claim 20 A method of presenting video information to a user, the method comprising: at a server computer: obtaining video information including a plurality of frames; determining a presence of at least one event in the video information; receiving a query for the presence of the at least one event in the video information; denoting a presence of the at least one event in the video information associated with at least one frame of the plurality of frames of the video information; and presenting to the user a visual representation of the at least one frame of the plurality of frames of the video information associated with the at least one event.