# US 10,484,439 B2 · Spectating data service for a spectating system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,484,439 B2
- Document ID: US10484439B2
- Status: Granted
- Kind: B2
- Application number: 14/755,944
- Filed: 2015-06-30
- Published: 2017-01-05
- Granted: 2019-11-19
- Earliest claimed: 2015-06-30
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Robert Harvey Oates, III
- Web page: https://patentgrep.com/patent/US10484439B2

## CPC

- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- A63F 13/86: Watching games played by other players
- H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06)
- H04N 21/2353: {specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata}
- H04N 21/472: End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content {(end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612)}
- H04N 21/47815: {Electronic shopping (payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12)}
- H04N 21/4781: {Games}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A spectating system that stores audiovisual (A/V) content for broadcasts and collects and stores spectating data including spectator interactions with the broadcasts and broadcast content, and that exposes the data and A/V content to consumers such as game developers, online merchants, or broadcasters. The spectating system temporally correlates the spectating data and A/V content so that consumers can review portions of video, audio, and/or text from broadcasts that may be related to particular spectator behaviors as indicated by respective spectating data. The spectating data may also be spatially mapped to the spectating interface so that consumers can determine spectator interest in and interactions with various broadcast content. The spectating data may include data specific to a broadcast, a plurality of broadcasts, a game, or a plurality of games.

## Claims

### Claim 1

A system, comprising:
one or more computing devices configured to implement a spectating system configured to:
obtain one or more broadcasts from one or more broadcaster devices, wherein each broadcast shows game play of a respective broadcaster participating as a player in a game executing on a game system;
generate broadcast content for the one or more broadcasts at least in part according to game metadata for the game received from the game system;
stream the one or more broadcasts and the broadcast content to a plurality of spectator devices;
record audiovisual (AN) content for the broadcasts;
obtain and store spectator interactions with the broadcasts and broadcast content as spectating data;
determine spectating data metrics indicating spectator states during the broadcasts based at least in part on analysis of the spectating data;
store correlations between the spectating data and the A/V content according to the spectating data metrics; and
expose the spectating data, the spectating data metrics and the A/V content to one or more spectating data consumer devices according to a spectating data interface.

### Claim 2 (depends on 1)

The system as recited in claim 1, wherein the spectating system is further configured to store at least part of the game metadata for the broadcasts as spectating data, wherein the game metadata includes one or more of game states for the game, game event data for the game, or data describing or representing game content.

### Claim 3 (depends on 1)

The system as recited in claim 1, wherein the broadcast content includes one or more of content overlaid on game video received from the broadcaster devices or UI elements provided to the plurality of spectator devices for display according to a spectating UI, and wherein the spectator interactions with the broadcast content indicate spectator interest in or selections of the respective broadcast content.

### Claim 4 (depends on 1)

The system as recited in claim 1, wherein the broadcast content includes user interface (UI) elements for viewing and purchasing virtual or physical objects including objects related to the game, and wherein the spectator interactions include interactions with the UI elements indicating spectator interest in or purchases of respective objects.

### Claim 5 (depends on 1)

The system as recited in claim 1, wherein the A/V content for a broadcast includes one or more of broadcast video, broadcast audio, or broadcast text chat.

### Claim 6 (depends on 1)

The system as recited in claim 1, wherein the spectating system is further configured to:
receive an indication of particular spectating data from a consumer device according to the spectating data interface;
determine a portion of the A/V content that is correlated to the particular spectating data; and
provide the correlated portion of the A/V content to the consumer device according to the spectating data interface.

### Claim 7 (depends on 1)

The system as recited in claim 1, wherein the spectating data interface includes one or more of an application programming interface (API) a user interface (UI), or a query interface.

### Claim 8

A method, comprising:
performing, by a spectating system implemented on one or more computing devices:
streaming one or more broadcasts received from one or more broadcaster devices to a plurality of spectator devices, wherein each broadcast includes game play of a respective broadcaster participating as a player in a game executing on a game system;
providing broadcast content to the spectator devices with the one or more broadcasts, wherein the broadcast content is determined at least in part according to game metadata for the game received from the game system;
obtaining and recording spectator interactions with the one or more broadcasts and respective broadcast content as spectating data;
determine spectating data metrics indicating spectator states during the one or more broadcasts based at least in part on analysis of the spectator interactions with the one or more broadcasts and the respective broadcast content;
storing correlations between the spectating data and the one or more broadcasts according to the spectating data metrics; and
exposing the spectating data and the spectating data metrics to one or more spectating data consumers according to a spectating data interface.

### Claim 9 (depends on 8)

The method as recited in claim 8, further comprising:
recording audiovisual (A/V) content for the one or more broadcasts; and
determining the correlations between the spectating data and the A/V content according to the spectating data metrics.

### Claim 10 (depends on 9)

The method as recited in claim 9, wherein the A/V content for a broadcast includes one or more of broadcast video, broadcast audio, or broadcast text chat.

### Claim 11 (depends on 8)

The method as recited in claim 8, further comprising:
recording audiovisual (A/V) content for the one or more broadcasts;
receiving an indication of particular spectating data from a consumer device according to the spectating data interface;
determining a portion of the A/V content that is correlated to the particular spectating data; and
providing the correlated portion of the A/V content to the consumer device according to the spectating data interface.

### Claim 12 (depends on 8)

The method as recited in claim 8, further comprising storing at least part of the game metadata for the broadcasts as spectating data, wherein the game metadata includes one or more of game states for the game, game event data for the game, or data describing or representing game content.

### Claim 13 (depends on 8)

The method as recited in claim 8, further comprising:
determining spectating statistics for the one or more broadcasts based on spectator participation in the broadcasts; and
storing the spectating statistics with the spectating data.

### Claim 14 (depends on 8)

The method as recited in claim 8, wherein the broadcast content includes one or more of content overlaid on game video received from the broadcaster devices or UI elements provided to the plurality of spectator devices for display according to a spectating UI, and wherein the spectator interactions with the broadcast content indicate spectator interest in or selections of the respective broadcast content.

### Claim 15 (depends on 8)

The method as recited in claim 8, wherein the broadcast content includes user interface (UI) elements for viewing and purchasing virtual or physical objects including objects related to the game, and wherein the spectator interactions include interactions with the UI elements indicating spectator interest in or purchases of respective objects.

### Claim 16 (depends on 8)

The method as recited in claim 8, further comprising:
analyzing the spectating data to determine spectating information for at least one of the broadcasts; and
providing the spectating information to at least one of the spectating data consumers according to the spectating data interface.

### Claim 17 (depends on 16)

The method as recited in claim 16, wherein the spectating information includes a heat map that indicates spectator interest in the broadcast and respective broadcast content based on the spectator interactions with the broadcast and respective broadcast content.

### Claim 18 (depends on 8)

The method as recited in claim 8, wherein the spectating data consumers include one or more of game developers, broadcasters, or online merchants.

### Claim 19 (depends on 8)

The method as recited in claim 8, wherein the spectating data interface includes one or more of an application programming interface (API), a user interface (UI), or a query interface.

### Claim 20 (depends on 8)

The method as recited in claim 8, wherein said obtaining and recording spectator interactions with the broadcasts and broadcast content comprises tracking and recording interactions with a spectating user interface (UI) of the spectating system on respective spectator devices.

### Claim 21 (depends on 20)

The method as recited in claim 20, wherein the spectating UI is a Web page presented on a Web browser, and wherein said obtaining and recording spectator interactions with the broadcasts and broadcast content further comprises tracking spectator navigation to one or more other Web pages from the spectating UI.

### Claim 22 (depends on 20)

The method as recited in claim 20, wherein tracking and recording interactions with the spectating UI on at least one spectator device comprises tracking and recording one or both of eye motion or head motion with respect to the spectating UI.

### Claim 23 (depends on 8)

The method as recited in claim 8, wherein the spectating data includes data specific to a broadcast, a plurality of broadcasts, a game, or a plurality of games.

### Claim 24

A non-transitory computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to implement a spectating data service for a spectating system configured to:
store spectating data including indications of spectator interactions with a plurality of broadcasts and respective broadcast content, wherein each broadcast includes game play of a respective broadcaster participating as a player in a game executing on a game system, and wherein the broadcast content is determined at least in part according to game metadata for the game received from the game system;
store audiovisual (A/V) content for the broadcasts, wherein the A/V content for a broadcast includes one or more of broadcast video, broadcast audio, or broadcast text chat;
analyze the spectating data and the A/V content to determine spectating data metrics indicating spectator states during the broadcasts;
expose the spectating data, the spectating data metrics and the A/V content to one or more spectating data consumers according to a spectating data interface;
receive a request for particular spectating data from a spectating data consumer according to the spectating data interface;
determine a portion of the A/V content that is correlated to the particular spectating data according to the spectating data metrics; and
provide the requested spectating data and the correlated portion of the A/V content to the consumer according to the spectating data interface.

### Claim 25 (depends on 24)

The non-transitory computer-accessible storage medium as recited in claim 24, wherein the spectating data service is further configured to:
analyze the spectating data to determine spectating information specific to a broadcast, a plurality of broadcasts, a game, or a plurality of games; and
provide the spectating information to at least one of the spectating data consumers according to the spectating data interface.