# US 2010/0166062 A1 · System and Method for Selecting a Video Encoding Format Based on Feedback Data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0166062 A1
- Document ID: US20100166062A1
- Status: Published application
- Kind: A1
- Application number: 12/538,090
- Filed: 2009-08-07
- Published: 2010-07-01
- Earliest claimed: 2002-12-10
- Assignees: OL2, Inc.
- Inventors: Stephen G. Perlman; Roger van der Laan; Timothy Cotter; Scott Furman; Robert McCool; Ian Buckley
- Web page: https://patentgrep.com/patent/US20100166062A1

## CPC

- H04L 65/752: adapting media to network capabilities
- H04N 19/164: Feedback from the receiver or from the transmission channel
- A63F 13/25: Output arrangements for video game devices
- A63F 13/32: using local area network [LAN] connections
- A63F 13/335: using Internet
- A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
- A63F 13/358: Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
- A63F 13/86: Watching games played by other players
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06)
- H04L 65/765: {intermediate}
- H04L 67/131: Protocols for games, networked simulations or virtual reality
- H04N 19/107: between spatial and temporal predictive coding, e.g. picture refresh
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/146: Data rate or code amount at the encoder output
- H04N 19/169: characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N 19/172: the region being a picture, frame or field
- H04N 19/188: {the unit being a video data packet, e.g. a network abstraction layer [NAL] unit}
- H04N 19/436: using parallelised computational arrangements
- H04N 19/61: in combination with predictive coding
- H04N 21/2343: involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)}
- H04N 21/2381: Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network {(transmission of MPEG streams over ATM H04L12/5601)}
- H04N 21/6377: directed to server {(one-way streaming services wherein the source is controlled by the destination H04L65/613)}
- H04N 21/658: Transmission by the client directed to the server
- H04N 21/6583: Acknowledgement
- H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection
- A63F 2300/407: Data transfer via internet
- A63F 2300/534: for network load management, e.g. bandwidth optimization, latency reduction
- A63F 2300/538: for performing operations on behalf of the game client, e.g. rendering
- A63F 2300/552: for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
- A63F 2300/572: Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
- A63F 2300/577: for watching a game played by other players
- A63F 2300/636: involving process of starting or resuming a game
- H04N 7/106: {for domestic distribution}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

A computer-implemented system and method for performing video compression are described. For example, a method according to one embodiment comprises: encoding a plurality of video frames or portions thereof according to a first encoding format; transmitting the plurality of encoded video frames or portions to a client device; receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has been successfully received and/or decoded; in response to detecting that one or more video frames or portions thereof have not been successfully received and/or decoded, determining a number of video frames or portions thereof which have not been successfully received and/or decoded and: (1) if the number of video frames or portions thereof which have not been successfully received and/or decoded is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or (2) if the number of video frames or portions thereof which have not been successfully received and/or decoded is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof; and transmitting the new video frame or portion thereof to the client device.

## Claims

### Claim 1

A computer-implemented method for performing video compression comprising:
encoding a plurality of video frames or portions thereof according to a first encoding format;
transmitting the plurality of encoded video frames or portions to a client device;
receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has been successfully received and/or decoded;
in response to detecting that one or more video frames or portions thereof have not been successfully received and/or decoded, determining a number of video frames or portions thereof which have not been successfully received and/or decoded and:
(1) if the number of video frames or portions thereof which have not been successfully received and/or decoded is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or
(2) if the number of video frames or portions thereof which have not been successfully received and/or decoded is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof; and
transmitting the new video frame or portion thereof to the client device.

### Claim 2 (depends on 1)

The method as in claim 1 further comprising:
determining latency associated with communicating with the client device; and
(1) if the latency is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or
(2) if the latency is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof.

### Claim 3 (depends on 2)

The method as in claim 2 wherein the frames or portions thereof comprise P-frames or P-tiles, respectively, when encoded according to the first encoding format and wherein the frames or portions thereof encoded comprise I-frames or I-tiles, respectively, when encoded according to the second encoding format.

### Claim 4 (depends on 1)

The method as in claim 1 wherein the feedback information comprises an indication that the frames or portions thereof have been successfully received and/or decoded at the client device.

### Claim 5 (depends on 1)

The method as in claim 1 wherein the feedback information comprises an indication that the frames or portions thereof have not been successfully received and/or decoded at the client device.

### Claim 6 (depends on 1)

The method as in claim 1 wherein the video frames comprise images of a video game being played at the client device.

### Claim 7 (depends on 1)

The method as in claim 1 further comprising:
decoding the encoded video frames or portion thereof at the client device; and
displaying images associated with each of the video frames or portions thereof on a display on the client device.

### Claim 8

A system comprising a memory for storing program code and a processor for processing the program code to perform the operations of:
encoding a plurality of video frames or portions thereof according to a first encoding format;
transmitting the plurality of encoded video frames or portions to a client device;
receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has been successfully received and/or decoded;
in response to detecting that one or more video frames or portions thereof have not been successfully received and/or decoded, determining a number of video frames or portions thereof which have not been successfully received and/or decoded and:
(1) if the number of video frames or portions thereof which have not been successfully received and/or decoded is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or
(2) if the number of video frames or portions thereof which have not been successfully received and/or decoded is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof; and
transmitting the new video frame or portion thereof to the client device.

### Claim 9 (depends on 8)

The system as in claim 8 wherein the processor processes the program code to determine latency associated with communicating with the client device; and
(1) if the latency is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or
(2) if the latency is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof.

### Claim 10 (depends on 9)

The system as in claim 9 wherein the frames or portions thereof comprise P-frames or P-tiles, respectively, when encoded according to the first encoding format and wherein the frames or portions thereof encoded comprise I-frames or I-tiles, respectively, when encoded according to the second encoding format.

### Claim 11 (depends on 8)

The system as in claim 8 wherein the feedback information comprises an indication that the frames or portions thereof have been successfully received and/or decoded at the client device.

### Claim 12 (depends on 8)

The system as in claim 8 wherein the feedback information comprises an indication that the frames or portions thereof have not been successfully received and/or decoded at the client device.

### Claim 13 (depends on 8)

The system as in claim 8 wherein the video frames comprise images of a video game being played at the client device.

### Claim 14 (depends on 8)

The system as in claim 8 comprising additional program code to cause the processor to perform the operations of:
decoding the encoded video frames or portion thereof at the client device; and
displaying images associated with each of the video frames or portions thereof on a display on the client device.

### Claim 15

A machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform the operations of:
encoding a plurality of video frames or portions thereof according to a first encoding format;
transmitting the plurality of encoded video frames or portions to a client device;
receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has been successfully received and/or decoded;
in response to detecting that one or more video frames or portions thereof have not been successfully received and/or decoded, determining a number of video frames or portions thereof which have not been successfully received and/or decoded and:
(1) if the number of video frames or portions thereof which have not been successfully received and/or decoded is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or
(2) if the number of video frames or portions thereof which have not been successfully received and/or decoded is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof; and
transmitting the new video frame or portion thereof to the client device.

### Claim 16 (depends on 15)

The machine-readable medium as in claim 15 further comprising program code to determine latency associated with communicating with the client device; and
(1) if the latency is above a specified threshold, then encoding a new video frame or portion thereof according to a second encoding format, the second encoding format comprising a format which is not dependent on previously-transmitted video frames or portions thereof; or
(2) if the latency is below a specified threshold, then encoding a new video frame or portion thereof according to the first encoding format, the new video frame encoded to be dependent on a last known successfully received video frame or portion thereof.

### Claim 17 (depends on 16)

The machine-readable medium as in claim 16 wherein the frames or portions thereof comprise P-frames or P-tiles, respectively, when encoded according to the first encoding format and wherein the frames or portions thereof encoded comprise I-frames or I-tiles, respectively, when encoded according to the second encoding format.

### Claim 18 (depends on 15)

The machine-readable medium as in claim 15 wherein the feedback information comprises an indication that the frames or portions thereof have been successfully received and/or decoded at the client device.

### Claim 19 (depends on 15)

The machine-readable medium as in claim 15 wherein the feedback information comprises an indication that the frames or portions thereof have not been successfully received and/or decoded at the client device.

### Claim 20 (depends on 15)

The machine-readable medium as in claim 15 wherein the video frames comprise images of a video game being played at the client device.

### Claim 21 (depends on 15)

The machine-readable medium as in claim 15 comprising additional program code to cause the machine to perform the operations of:
decoding the encoded video frames or portion thereof at the client device; and
displaying images associated with each of the video frames or portions thereof on a display on the client device.
