# US 2021/0243239 A1 · Independent control of interactive streaming media

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0243239 A1
- Document ID: US20210243239A1
- Status: Published application
- Kind: A1
- Application number: 17/302,053
- Filed: 2021-04-22
- Published: 2021-08-05
- Earliest claimed: 2015-07-28
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Dov Shimon Zimring; Ali Nadaff; Michael Jon Sundermeyer; Rishi Chandra; John Affaki; Sarah Walter; Rob McCool; Majd Bakar
- Web page: https://patentgrep.com/patent/US20210243239A1

## CPC

- H04L 65/612: for unicast
- A63F 13/23: for interfacing with the game device, e.g. specific interfaces between game controller and console
- A63F 13/26: having at least one additional display device, e.g. on the game controller or outside a game booth
- A63F 13/335: using Internet
- A63F 13/352: involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
- A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
- A63F 13/358: Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
- A63F 13/42: by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- A63F 13/428: involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
- A63F 13/79: involving player-related data, e.g. identities, accounts, preferences or play histories
- H04L 65/1094: Inter-user-equipment sessions transfer or sharing
- H04L 65/613: for the control of the source by the destination (control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637)
- H04L 65/762: {at the source (reformatting of additional data in video distribution servers H04N21/2355)}
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

In one general aspect, a method can include receiving, at a server computing device, a launch request to launch and stream media content pertaining to a game session, the launch request being provided by a mobile device, selecting, using the launch request, a media host configured to stream the media content, verifying a user associated with the mobile device and launching the game session for streaming to the user on a device other than the mobile device. In addition, the method can include during the game session and while streaming the media content, receiving a plurality of real time control requests from the mobile device, and executing the control request by compositing, in real time, the at least one change and the media content and to produce a composite display and transmitting, to the device other than the mobile device, the encoded composite display for streaming in real time.

## Claims

### Claim 1

A system comprising:
a mobile device configured to:
wirelessly connect to an input device;
select at least one content server computing device to host a streaming session with at least one media player device;
establish a streaming session with the at least one content server computing device;
receive input initiated by, and transmitted remotely from the input device, the input including instructions to control the content provided by the at least one content server computing device;
generate, based on the received input, a representation of the input; and
relaying, to the at least one content server computing device, the generated representation of the input, wherein the generated representation of the input includes a plurality of redundant packets that replicate the received input.

### Claim 2 (depends on 1)

The system of claim 1, wherein the representation of the input is used to control content displayed on the at least one media player device and provided by the at least one content server computing device.

### Claim 3 (depends on 2)

The system of claim 2, wherein controlling the content displayed on the at least one media player device comprises:
causing execution of the input, received from the mobile device, on streaming media content by triggering execution, in real time, of at least one change to the media content to produce a composite display;
causing encoding of the composite display for transmission to a device other than the mobile device; and
causing transmission, from the mobile device to the device other than the mobile device, of the encoded composite display.

### Claim 4 (depends on 3)

The system of claim 3, wherein the device other than the mobile device comprises a media player coupled to a display device for streaming for display the encoded composite display in real time.

### Claim 5 (depends on 1)

The system of claim 1, wherein the input device is a second mobile device.

### Claim 6 (depends on 1)

The system of claim 1, wherein each redundant packet includes an identical sequence identifier.

### Claim 7 (depends on 1)

The system of claim 1, wherein the mobile device is further configured to:
transmit a reset protocol to prevent replay of commands associated with the plurality of redundant packets, in response to determining that the plurality of redundant packets were not received by the at least one content server computing system.

### Claim 8

A computer-implemented method comprising:
causing, at a mobile device, an input device to wirelessly connect to the mobile device;
selecting, by the mobile device, at least one content server computing device to host a streaming session with at least one media player device;
initiating, by the mobile device, a streaming session with the at least one content server computing device;
receiving, at the mobile device, input initiated by, and transmitted remotely from the input device, the input including instructions to control the content provided by the at least one content server computing device;
generating, by the mobile device and based on the received input, a representation of the input; and
relaying, from the mobile device and to the at least one content server computing device, the generated representation of the input, wherein the generated representation of the input includes a plurality of redundant packets that replicate the received input.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the representation of the input is used to control content displayed on the at least one media player device and provided by the at least one content server computing device.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein controlling the content displayed on the at least one media player device comprises:
causing execution of the input, received from the mobile device, on streaming media content by triggering execution, in real time, of at least one change to the media content to produce a composite display;
causing encoding of the composite display for transmission to a device other than the mobile device; and
causing transmission, from the mobile device to the device other than the mobile device, of the encoded composite display for streaming in real time.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the device other than the mobile device comprises a media player coupled to a display device for streaming for display the encoded composite display in real time.

### Claim 12 (depends on 8)

The computer-implemented method of claim 8, wherein the input device is a second mobile device.

### Claim 13 (depends on 8)

The computer-implemented method of claim 8, wherein each redundant packet includes an identical sequence identifier.

### Claim 14 (depends on 8)

The computer-implemented method of claim 8, wherein the mobile device is further configured to:
transmit a reset protocol to prevent replay of commands associated with the plurality of redundant packets, in response to determining that the plurality of redundant packets were not received by the at least one content server computing system.

### Claim 15

A computer program product tangibly embodied on a non-transitory computer-readable medium and comprising instructions that, when executed, are configured to cause at least one processor to:
cause, at a mobile device, an input device to wirelessly connect to the mobile device;
select, by the mobile device, at least one content server computing device to host a streaming session with at least one media player device;
initiate, by the mobile device, a streaming session with the at least one content server computing device;
receive, at the mobile device, input initiated by, and transmitted remotely from the input device, the input including instructions to control the content provided by the at least one content server computing device;
generate, by the mobile device and based on the received input, a representation of the input; and
relay, from the mobile device and to the at least one content server computing device, the generated representation of the input, wherein the generated representation of the input includes a plurality of redundant packets that replicate the received input.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the representation of the input is used to control content displayed on the at least one media player device and provided by at least one content server computing device.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein controlling the content displayed on the at least one media player device comprises:
causing execution of the input, received from the mobile device, on streaming media content by triggering execution, in real time, of at least one change to the media content to produce a composite display;
causing encoding of the composite display for transmission to a device other than the mobile device; and
causing transmission, from the mobile device to the device other than the mobile device, of the encoded composite display for streaming in real time.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein the input device is a second mobile device.

### Claim 19 (depends on 15)

The computer program product of claim 15, wherein each redundant packet includes an identical sequence identifier.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein the mobile device is further configured to:
transmit a reset protocol to prevent replay of commands associated with the plurality of redundant packets, in response to determining that the plurality of redundant packets were not received by at least one content server computing system.