# US 2021/0344736 A1 · MEASURING QUALITY-OF-EXPERIENCE (QoE) FOR VIRTUAL REALITY (VR) STREAMING CONTENT

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0344736 A1
- Document ID: US20210344736A1
- Status: Published application
- Kind: A1
- Application number: 17/374,565
- Filed: 2021-07-13
- Published: 2021-11-04
- Earliest claimed: 2020-01-15
- Assignees: Charter Communications Operating, LLC
- Applicants: Charter Communications Operating, LLC
- Inventors: Matthew Ringenberg; Dhananjay Lal; Dell Wolfensparger; Andrew Ip
- Web page: https://patentgrep.com/patent/US20210344736A1

## CPC

- H04L 65/80: Responding to QoS
- G06F 3/012: {Head tracking input arrangements}
- H04L 41/5009: Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
- H04L 43/0829: Packet loss
- H04L 43/087: Jitter
- H04L 65/1059: End-user terminal functionalities specially adapted for real-time communication
- H04L 65/61: for supporting one-way streaming services, e.g. Internet radio
- H04L 65/70: Media network packetisation
- H04N 13/327: Calibration thereof
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Measuring quality-of-experience (QoE) for virtual reality (VR) streaming content is disclosed. A network computing device receives a client-side VR stream capture and a client pose data set that are generated by a client computing device based on a VR content and one or more induced network impairments (e.g., latency, packet loss, and/or jitter, as non-limiting examples). Using the same VR content and the client pose data set, the network computing device generates a source VR stream capture that is not subjected to the one or more induced network impairments. The network computing device performs a frame-by-frame comparison of the client-side VR stream capture and the source VR stream capture. Based on the frame-by-frame comparison, the network computing device generates a QoE metric that indicates a degree of degradation of the client-side VR stream capture relative to the source VR stream capture.

## Claims

### Claim 1

A method, comprising:
streaming, by a client computing device via a network connection to a virtual reality (VR) head-mounted display device (HMDD) communicatively coupled to the client computing device, VR content received from a VR server of a network computing device;
recording, by the client computing device, movements of the HMDD based on the VR content while streaming the VR content from the VR server to the HMDD as a client pose data set;
recording, by the client computing device, visual output of the HMDD based on the VR content as a client-side VR stream capture; and
transmitting, by the client computing device, the client-side VR stream capture and the client pose data set to the network computing device.

### Claim 2 (depends on 1)

The method of claim 1, wherein recording the movements of the HMDD while streaming the VR content from the VR server to the HMDD as the client pose data set comprises recording the movements of the HMDD while the HMDD is selectively rotated by a computer-controlled turntable to which the HMDD is affixed.

### Claim 3 (depends on 2)

The method of claim 2, wherein the HMDD is selectively rotated by the computer-controlled turntable according to a rotation-and-velocity script.

### Claim 4 (depends on 1)

The method of claim 1, wherein the client pose data set comprises data indicating one or more of a position of the HMDD and an orientation of the HMDD.

### Claim 5 (depends on 1)

The method of claim 1, further comprising injecting, by the network computing device, one or more induced network impairments into the network connection while streaming the VR content;
wherein:
recording the movements of the HMDD is further based on the one or more induced network impairments; and
recording the visual output of the HMDD is further based on the one or more induced network impairments.

### Claim 6 (depends on 5)

The method of claim 5, wherein the one or more induced network impairments comprises one or more of network latency, packet loss, and jitter.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
receiving, by the network computing device from the client computing device, the client-side VR stream capture and the client pose data set;
generating a source VR stream capture based on the VR content and the client pose data set without the one or more induced network impairments;
performing a frame-by-frame comparison of the client-side VR stream capture and the source VR stream capture; and
generating a quality-of-experience (QoE) metric indicating a degree of degradation of the client-side VR stream capture relative to the source VR stream capture, based on the frame-by-frame comparison.

### Claim 8 (depends on 7)

The method of claim 7, wherein the QoE metric comprises a Structural Similarity Index Metric (SSIM).

### Claim 9 (depends on 7)

The method of claim 7, further comprising identifying, based on the QoE metric, a network location from which to stream the VR content.

### Claim 10

A computing system, comprising:
a client computing device comprising:
a first memory; and
a first processor device coupled to the first memory and configured to:
stream, via a network connection to a virtual reality (VR) head-mounted display device (HMDD) communicatively coupled to the client computing device, VR content received from a VR server of a network computing device;
record movements of the HMDD based on the VR content while streaming the VR content from the VR server to the HMDD as a client pose data set;
record visual output of the HMDD based on the VR content as a client-side VR stream capture; and
transmit the client-side VR stream capture and the client pose data set to the network computing device.

### Claim 11 (depends on 10)

The computing system of claim 10, wherein the first processor device is configured to record the movements of the HMDD while streaming the VR content from the VR server to the HMDD as the client pose data set by being configured to record the movements of the HMDD while the HMDD is selectively rotated by a computer-controlled turntable to which the HMDD is affixed.

### Claim 12 (depends on 11)

The computing system of claim 11, wherein the HMDD is selectively rotated by the computer-controlled turntable according to a rotation-and-velocity script.

### Claim 13 (depends on 10)

The computing system of claim 10, wherein the client pose data set comprises data indicating one or more of a position of the HMDD and an orientation of the HMDD.

### Claim 14 (depends on 10)

The computing system of claim 10, further comprising:
the network computing device comprising:
a second memory; and
a second processor device coupled to the second memory and configured to inject one or more induced network impairments into the network connection while streaming the VR content;
wherein:
the first processor device is configured to record the movements of the HMDD further based on the one or more induced network impairments; and
the first processor device is configured to record the visual output of the HMDD further based on the one or more induced network impairments.

### Claim 15 (depends on 14)

The computing system of claim 14, wherein the one or more induced network impairments comprises one or more of network latency, packet loss, and jitter.

### Claim 16 (depends on 14)

The computing system of claim 14, wherein the second processor device is further configured to:
receive, from the client computing device, the client-side VR stream capture and the client pose data set;
generate a source VR stream capture based on the VR content and the client pose data set without the one or more induced network impairments;
perform a frame-by-frame comparison of the client-side VR stream capture and the source VR stream capture; and
generate a quality-of-experience (QoE) metric indicating a degree of degradation of the client-side VR stream capture relative to the source VR stream capture, based on the frame-by-frame comparison.

### Claim 17 (depends on 16)

The computing system of claim 16, wherein the QoE metric comprises a Structural Similarity Index Metric (SSIM).

### Claim 18 (depends on 16)

The computing system of claim 16, wherein the second processor device is further configured to identify, based on the QoE metric, a network location from which to stream the VR content.

### Claim 19

A computer program product stored on a non-transitory computer-readable storage medium and including computer-executable instructions configured to cause a processor device to:
stream, via a network connection to a virtual reality (VR) head-mounted display device (HMDD) communicatively coupled to a client computing device, VR content received from a VR server of a network computing device;
record movements of the HMDD based on the VR content while streaming the VR content from the VR server to the HMDD as a client pose data set;
record visual output of the HMDD based on the VR content as a client-side VR stream capture; and
transmit the client-side VR stream capture and the client pose data set to the network computing device.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the computer-executable instructions cause the processor device to record the movements of the HMDD while streaming the VR content from the VR server to the HMDD as the client pose data set by causing the processor device to record the movements of the HMDD while the HMDD is selectively rotated by a computer-controlled turntable to which the HMDD is affixed.