# US 2019/0173929 A1 · Method and apparatus for an adaptive video-aware streaming architecture with cloud-based prediction and elastic rate control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0173929 A1
- Document ID: US20190173929A1
- Status: Published application
- Kind: A1
- Application number: 16/027,975
- Filed: 2018-07-05
- Published: 2019-06-06
- Earliest claimed: 2017-07-05
- Assignees: Cinova Media
- Applicants: Cinova Media
- Inventors: Domenico Guardini; Pierre Seigneurbieux
- Web page: https://patentgrep.com/patent/US20190173929A1

## CPC

- H04L 65/70: Media network packetisation
- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- H04L 65/613: for the control of the source by the destination (control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637)
- H04L 65/752: adapting media to network capabilities
- H04L 65/765: {intermediate}
- H04L 65/80: Responding to QoS
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A method and apparatus for an adaptive video-aware streaming architecture are disclosed. The architecture may include cloud-based prediction and elastic rate control.

## Claims

### Claim 1

A streaming virtual reality data system, comprising:
a virtual reality data backend having a plurality of geographically distributed server computers;
a plurality of virtual reality devices connected to the virtual reality data backend, each virtual reality device having a head mounted display that generates a set of prediction data about movement of the head mounted display, each virtual reality device connecting to a particular server computer of the virtual reality data backend so that a round-trip time for virtual reality data between the virtual reality device and the particular server computer is less than a prediction time; and
each server computer having a prediction engine that generates a set of characteristics for each virtual reality device coupled to the virtual reality data backend and predicts a field of view for each virtual reality device based in part on the set of generated characteristics.

### Claim 2 (depends on 1)

The system of claim 1, wherein the set of generated characteristics further comprises one or more of prediction quality on a per device type, a user heat-map of specific virtual reality data, network characteristics per geo-location impacting the average, min, max and variance of the round-trip delay and a rate of packet loss per user.

### Claim 3 (depends on 1)

The system of claim 1, wherein the prediction engine further comprises a machine learning element that generates the set of characteristics and predicts the field of view for each virtual reality device based in part on the set of generated characteristics.

### Claim 4 (depends on 1)

The system of claim 1, wherein each virtual reality device connects to a particular server computer of the virtual reality data backend so that a round-trip time for virtual reality data between the virtual reality device and the particular server computer is equal or smaller compared to the prediction time.

### Claim 5 (depends on 1)

The system of claim 1, wherein the backend further comprises an error recovery engine that recovers from virtual reality data packet loss.

### Claim 6 (depends on 5)

The system of claim 5, wherein the error recovery engine determines a type of the frame of virtual reality data that is lost and resends the lost data packet when the lost data packet contains a key frame.

### Claim 7 (depends on 1)

The system of claim 1, wherein the backend further comprises an adaptive rate control engine that adjusts a number of frames in a video buffer based on the set of generated characteristics.

### Claim 8 (depends on 7)

The system of claim 7, wherein the adjusted number of frames in the video buffer, for each virtual reality device, is determined based on a prediction quality characteristics value subtracted from a round trip time characteristic reflecting a time that virtual reality data travels between each virtual reality device and the backend.

### Claim 9 (depends on 1)

The system of claim 1, wherein the backend further comprises an adaptive rate control engine that encodes a different total number of field of views for each bitrate range or profile.

### Claim 10 (depends on 1)

The system of claim 1, wherein the backend further comprises an adaptive rate control engine that adapts a bitrate to a movement characteristics of each virtual reality device headset.

### Claim 11

A method for streaming virtual reality field of view data, the method comprising:
providing a virtual reality data backend having a plurality of geographically distributed server computers and a plurality of virtual reality devices connected to the virtual reality data backend, each virtual reality device having a head mounted display that generates a set of prediction data about movement of the head mounted display, each virtual reality device connecting to a particular server computer of the virtual reality data backend so that a round-trip time for virtual reality data between the virtual reality device and the particular server computer is less than a prediction time;
generating a set of characteristics for each virtual reality device coupled to the virtual reality data backend; and
predicting a field of view for each virtual reality device based in part on the set of generated characteristics.

### Claim 12 (depends on 11)

The method of claim 11, wherein the set of generated characteristics further comprises one or more of prediction quality on a per device type, a user heat-map of specific virtual reality data, network characteristics per geo-location impacting the average, min, max and variance of the round-trip delay and a rate of packet loss per user.

### Claim 13 (depends on 11)

The method of claim 11 further comprising recovering from virtual reality data packet loss.

### Claim 14 (depends on 13)

The method of claim 13, wherein recovering from virtual reality data packet loss further comprises determining a type of the frame of virtual reality data that is lost and resending the lost data packet when the lost data packet contains a key frame.

### Claim 15 (depends on 11)

The method of claim 11 further comprising adjusting a number of frames in a video buffer based on the set of generated characteristics.

### Claim 16 (depends on 15)

The method of claim 15, wherein adjusting the number of frames further comprises adjusting the number of frames based on a prediction quality characteristics value subtracted from a round trip time characteristic reflecting a time that virtual reality data travels between each virtual reality device and the backend.

### Claim 17 (depends on 11)

The method of claim 11 further comprising encoding a different total number of field of views for each bitrate range or profile.

### Claim 18 (depends on 11)

The method of claim 11 further comprising adapting a bitrate to a movement characteristics of each virtual reality device headset.
