# US 10,864,447 B1 · Highlight presentation interface in a game spectating system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,864,447 B1 - Document ID: US10864447B1 - Status: Granted - Kind: B1 - Application number: 14/754,584 - Filed: 2015-06-29 - Granted: 2020-12-15 - Earliest claimed: 2015-06-29 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Michael Anthony Willette; Patrick Gilmore; Ethan Zane Evans; Michael Martin George; Michael Anthony Frazzini - Web page: https://patentgrep.com/patent/US10864447B1 ## CPC - A63F 13/86: Watching games played by other players - A63F 13/35: Details of game servers - A63F 13/58: by computing conditions of game characters, e.g. stamina, strength, motivation or energy level - H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/44204: {Monitoring of content usage, e.g. the number of times a movie has been viewed, copied or the amount which has been watched (monitoring of user activities for profile generation for accessing a video database G06F16/739; protecting generic digital content where the protection is independent of the precise nature of the content G06F21/10; arrangements for monitoring the use made of the broadcast services in broadcast systems H04H60/31)} - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/4781: {Games} - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04N 21/8549: Creating video summaries, e.g. movie trailer {(retrieval in video databases by using presentations in form of a video summary G06F16/739)} - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods and apparatus for presenting highlights from broadcasts in spectating environments. A highlight service determines highlight events in the broadcasts, and extracts highlight segments from the broadcasts according to the determined events. Highlight reels may be created from the highlight segments according to highlight selection criteria (genre, game title, broadcaster, time period, etc.) and/or spectator preferences. The highlight service may provide a highlight user interface (UI) via which spectators may selectively view the highlights and highlight reels. The highlight service may generate highlight reels targeted to particular spectators or groups of spectators according to spectators' specified or otherwise determined preferences, demographics, viewing history, and so on. ## Claims ### Claim 1 A system, comprising: one or more computing devices configured to implement a game spectating service configured to stream a plurality of broadcasts received from a plurality of broadcaster devices to a plurality of spectator devices, wherein each broadcast shows game play of a respective broadcaster participating as a player in a respective game executing on one of a plurality of game systems; one or more computing devices configured to implement a highlight service configured to: determine a plurality of highlight segments from the plurality of broadcasts according to highlight data received from the plurality of broadcaster devices, wherein each highlight segment includes a portion of a respective broadcast showing a notable event from the respective broadcast, wherein the highlight data is determined by respective broadcaster devices of the plurality of broadcaster devices, and wherein the highlight data indicates a plurality of notable events in the plurality of broadcasts; select a plurality of general sets of highlight segments from the plurality of highlight segments, wherein each of the plurality of general sets is selected according to one or more highlight selection criteria; for at least one of the spectator devices: obtain viewing history for a spectator associated with the spectator device; determine highlight segment metrics for previously viewed highlight segments indicated in the viewing history, the highlight segment metrics determined based at least in part on audio and/or text analysis of inputs provided by the respective broadcasters or respective spectators during recording of the previously viewed highlight segments; select particular ones of the plurality of general sets of highlight segments based at least in part on the spectator's viewing history; determine one or more targeted sets of highlight segments from the plurality of highlight segments based at least in part on identifying similar highlight segments of the plurality of highlight segments according to the highlight segment metrics for the spectator's viewing history; and provide access to the particular ones of the plurality of general sets of highlight segments and the determined one or more targeted sets of highlight segments to the spectator via a highlight user interface (UI) on the spectator device. ### Claim 2 (depends on 1) The system as recited in claim 1, wherein the highlight service is further configured to stream at least one of the one or more targeted sets of highlight segments to the spectator device for display via the highlight UI. ### Claim 3 (depends on 1) The system as recited in claim 1, wherein the highlight service is further configured to stream a particular one of the one or more targeted sets of highlight segments to the spectator device for display on the highlight UI in response to spectator input to the highlight UI selecting the set of highlight segments. ### Claim 4 (depends on 1) The system as recited in claim 1, wherein the one or more targeted sets of highlight segments from the plurality of highlight segments are determined further based at least in part on the spectator's highlight viewing preferences including indications of one or more of game genres, game titles, broadcasters, types of highlights, or time periods of interests to the respective spectator. ### Claim 5 (depends on 4) The system as recited in claim 4, wherein, to obtain highlight viewing preferences for a spectator associated with the spectator device, the highlight service is further configured to receive spectator input from the spectator device via the highlight UI, wherein the spectator input indicates at least some of the highlight viewing preferences for the spectator associated with the respective spectator device. ### Claim 6 (depends on 1) The system as recited in claim 1, wherein the highlight service is further configured to: for each of the plurality of general sets, generate a highlight reel including at least some of the highlight segments in the respective general set of highlight segments. ### Claim 7 (depends on 1) The system as recited in claim 1, wherein the one or more highlight selection criteria include one or more of game genres, game titles, broadcasters, types of highlights, or time periods. ### Claim 8 (depends on 1) The system as recited in claim 1, wherein the highlight service is further configured to: analyze the highlight data to determine game highlight statistics indicating one or more of a number, density, type, quality, and/or ratings of highlight segments detected in each of the plurality of broadcasts; determine whether to present a particular broadcast via the highlight UI on the spectator device based at least in part on the game highlight statistics; and present the particular broadcast via the highlight UI on the spectator device. ### Claim 9 A method, comprising: performing, by a game spectating system implemented on one or more computing devices: streaming a plurality of broadcasts received from a plurality of broadcaster devices to a plurality of spectator devices, wherein each broadcast shows video of a respective broadcaster; determining a plurality of highlight segments from the plurality of broadcasts according to highlight data received from the plurality of broadcaster devices, wherein each highlight segment includes a portion of a respective broadcast showing a notable event from the respective broadcast, wherein the highlight data is determined by respective broadcaster devices of the plurality of broadcaster devices, and wherein the highlight data indicates a plurality of notable events in the plurality of broadcasts; selecting a plurality of general sets of highlight segments from the plurality of highlight segments, wherein each of the plurality of general sets is selected according to one or more highlight selection criteria; obtaining viewing history for one or more spectators associated with the spectator devices; determining highlight segment metrics for previously viewed highlight segments indicated in the obtained viewing history, the highlight segment metrics determined based at least in part on audio and/or text analysis of inputs provided by the respective broadcasters or respective spectators during recording of the previously viewed highlight segments; selecting particular ones of the plurality of general sets of highlight segments based at least in part on the spectator's viewing history; determining one or more targeted sets of highlight segments from the plurality of highlight segments based at least in part on identifying similar highlight segments of the plurality of highlight segments according to the highlight segment metrics for the obtained viewing history; and providing access to the particular ones of the plurality of general sets of highlight segments and the determined one or more targeted sets of highlight segments to at least one of the one or more spectators via a highlight user interface (UI) on respective spectator devices. ### Claim 10 (depends on 9) The method as recited in claim 9, wherein at least one broadcast shows game play of a respective broadcaster participating as a player in a game executing on one of a plurality of game systems. ### Claim 11 (depends on 9) The method as recited in claim 9, further comprising streaming at least one of the one or more targeted sets of highlight segments to at least one of the spectator devices for display via the highlight UI. ### Claim 12 (depends on 9) The method as recited in claim 9, further comprising: receiving highlight selection input to the highlight UI from one of the spectator devices, the highlight selection input selecting a particular one of the one of the one or more targeted sets of highlight segments; and streaming the selected set of highlight segments to the respective spectator device for display on the highlight UI in response to the highlight selection input. ### Claim 13 (depends on 9) The method as recited in claim 9, wherein determining the one or more targeted sets of highlight segments from the plurality of highlight segments is further based at least in part on highlight viewing preferences including indications of one or more of game genres, game titles, broadcasters, types of highlights, or time periods of interests to the respective spectators. ### Claim 14 (depends on 9) The method as recited in claim 9, wherein determining the plurality of highlight segments comprises identifying respective start points and end points to provide context around the respective notable events of the respective highlight segments according to a preferred window length. ### Claim 15 (depends on 9) The method as recited in claim 9, wherein said determining one or more sets of highlight segments from the plurality of highlight segments comprises locating one or more other highlight segments that are similar to a selected highlight segment. ### Claim 16 (depends on 9) The method as recited in claim 9, further comprising obtaining highlight viewing preferences and viewing history for one or more spectators associated with the spectator devices comprising: receiving spectator input from the spectator devices via the highlight UI, wherein the spectator input from a particular one of the spectator devices indicates at least some of the highlight viewing preferences for a spectator associated with the respective spectator device; and receiving the viewing history from one or more spectator profiles for the one or more spectators. ### Claim 17 (depends on 9) The method as recited in claim 9, further comprising: for each of the plurality of general sets, generating a highlight reel including at least two of the highlight segments from the respective general set. ### Claim 18 (depends on 9) The method as recited in claim 9, wherein the one or more highlight selection criteria include one or more of game genres, game titles, broadcasters, types of highlights, or time periods. ### Claim 19 (depends on 17) The method as recited in claim 17, further comprising streaming at least one of the highlight reels to at least one of the plurality of spectator devices for display via the highlight UI. ### Claim 20 (depends on 9) The method as recited in claim 9, further comprising: determining an interesting or exciting broadcast according to analysis of highlight segment metadata from the plurality of broadcasts; and providing an indication of the interesting or exciting broadcast to at least one of the plurality of spectator devices for display via the highlight UI. ### Claim 21 (depends on 20) The method as recited in claim 20, wherein the highlight segment metadata includes one or more of number, density, type, quality, or ratings of highlight segments in respective broadcasts of the plurality of broadcasts. ### Claim 22 A non-transitory computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to implement a highlight service for a game spectating system, the highlight service configured to: determine a plurality of highlight segments from a plurality of broadcasts received from a plurality of broadcaster devices and streamed to a plurality of spectator devices according to highlight data received from the plurality of broadcaster devices, wherein each broadcast shows game play of a respective broadcaster participating as a player in a game executing on one of a plurality of game systems, wherein each highlight segment includes a portion of a respective broadcast showing a notable event from a respective game, wherein the highlight data is determined by respective broadcaster devices of the plurality of broadcaster devices, and wherein the highlight data indicates a plurality of notable events in the plurality of broadcasts; select a plurality of general sets of highlight segments from the plurality of highlight segments, wherein each of the plurality of general sets is selected according to one or more highlight selection criteria; determine viewing history for one or more spectators associated with the spectator devices; select particular ones of the plurality of general sets of highlight segments based at least in part on the spectator's viewing history; determine highlight segment metrics for previously viewed highlight segments indicated in the spectator's viewing history, the highlight segment metrics determined based at least in part on audio and/or text analysis of inputs provided by the respective broadcasters or respective spectators during recording of the previously viewed highlight segments; determine one or more targeted sets of highlight segments from the plurality of highlight segments from the plurality of broadcasts received from the plurality of broadcaster devices based at least in part on identifying similar highlight segments of the plurality of highlight segments according to the highlight segment metrics for the determined viewing history; and provide access to the particular ones of the plurality of general sets of highlight segments and the determined one or more targeted sets of highlight segments to at least one of the one or more spectators via a highlight user interface (UI) on respective spectator devices. ### Claim 23 (depends on 22) The non-transitory computer-accessible storage medium as recited in claim 22, wherein the highlight service is further configured to: receive highlight selection input to the highlight UI from one of the spectator devices, the highlight selection input selecting a particular one of the one of the one or more sets of highlight segments; and stream the selected set of highlight segments to the respective spectator device for display on the highlight UI in response to the highlight selection input. ### Claim 24 (depends on 22) The non-transitory computer-accessible storage medium as recited in claim 22, wherein, to determine highlight viewing preferences for a particular spectator associated with a particular spectator device, the highlight service is further configured to receive spectator input from the spectator device via the highlight UI, wherein the spectator input indicates at least some of the highlight viewing preferences for the spectator associated with the respective spectator device. ### Claim 25 (depends on 22) The non-transitory computer-accessible storage medium as recited in claim 22, wherein the one or more highlight selection criteria include one or more of game genres, game titles, broadcasters, types of highlights, or time periods, and wherein the highlight service is further configured to: for each of the plurality of general sets, generate a highlight reel including at least two of the highlight segments from the respective general set; and stream at least one of the highlight reels to at least one of the plurality of spectator devices for display via the highlight UI.