# US 2017/0251204 A1 · Independent multi-resolution coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0251204 A1
- Document ID: US20170251204A1
- Status: Published application
- Kind: A1
- Application number: 15/195,439
- Filed: 2016-06-28
- Published: 2017-08-31
- Earliest claimed: 2016-02-26
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Ajit Gupte; Ajit Rao; Mina Ayman Saleh Yanni Makar
- Web page: https://patentgrep.com/patent/US20170251204A1

## CPC

- H04N 13/315: the parallax barriers being time-variant
- H04N 13/178: Metadata, e.g. disparity information
- H04N 13/398: Synchronisation thereof; Control thereof
- H04N 19/146: Data rate or code amount at the encoder output
- H04N 19/154: Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion (use of rate-distortion criteria H04N19/147)
- H04N 19/167: Position within a video image, e.g. region of interest [ROI]
- H04N 19/33: in the spatial domain
- H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence)
- H04N 19/59: involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
- H04N 19/597: specially adapted for multi-view video sequence encoding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In one example, a device for retrieving video data includes a display interface coupled to a display, a memory configured to store video data, and one or more processors configured to determine a plurality of regions of video data to be displayed via the display, retrieve video data having a first quality for a first subset of the plurality of regions at which a visual focus of a user is directed, retrieve video data having a second quality for a second subset of the plurality of regions that neighbor the first subset of the plurality of regions, wherein the second quality is lower than the first quality, and retrieve video data having a third quality for a third subset of the plurality of regions that is outside the first subset and the second subset, wherein the third quality is lower than the second quality.

## Claims

### Claim 1

A method of retrieving video data, the method comprising:
determining a plurality of regions of video data to be displayed via a display;
retrieving video data having a first quality for a first subset of the plurality of regions at which a visual focus of a user is directed;
retrieving video data having a second quality for a second subset of the plurality of regions that neighbor the first subset of the plurality of regions, wherein the second quality is lower than the first quality; and
retrieving video data having a third quality for a third subset of the plurality of regions that is outside the first subset and the second subset, wherein the third quality is lower than the second quality.

### Claim 2 (depends on 1)

The method of claim 1,
wherein retrieving the video data having the first quality comprises retrieving video data having a first spatial resolution,
wherein retrieving the video data having the second quality comprises retrieving video data having a second spatial resolution that is lower than the first spatial resolution, and wherein the video data for the second subset of the plurality of regions is coded independently of the video data having the first spatial resolution, and
wherein retrieving the video data having the third quality comprises retrieving video data having a third spatial resolution that is lower than the second spatial resolution, and wherein the video data for the third subset of the plurality of regions is coded independently of the video data having the first spatial resolution and the video data having the second spatial resolution.

### Claim 3 (depends on 1)

The method of claim 1,
wherein retrieving the video data having the first quality comprises retrieving one or more base layers of the video data for the first subset of the plurality of regions and a first set of one or more enhancement layers of the video data for the first subset of the plurality of regions,
wherein retrieving the video data having the second quality comprises retrieving one or more of the base layers of the video data for the second subset of the plurality of regions and a second set of one or more enhancement layers of the video data for the second subset of the plurality of regions, and
wherein retrieving the video data having the third quality comprises retrieving one or more of the base layers of the video data for the third subset of the plurality of regions and not retrieving any enhancement layers for the third subset of the plurality of regions.

### Claim 4 (depends on 1)

The method of claim 1,
wherein retrieving the video data having the first quality comprises retrieving a first set of one or more representations of the video data for the first subset of the plurality of regions having first bitrates,
wherein retrieving the video data having the second quality comprises retrieving a second set of one or more representations of the video data for the second subset of the plurality of regions having second bitrates lower than the first bitrates, and
wherein retrieving the video data having the third quality comprises retrieving a third set of one or more representations of the video data for the third subset of the plurality of regions having third bitrates lower than the second bitrates.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first subset of the plurality of regions comprises two or more regions, wherein video data is separately available for each of the two or more regions, the method further comprising:
determining whether a single set of video data is available for each of the two or more regions; and
when the single set of video data is available for each of the two or more regions, retrieving the single set of video data without retrieving the video data that is separately available for each of the two or more regions.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
determining that the focus of the user has been redirected to a fourth subset of the plurality of regions;
in response to the determination that the focus of the user has been redirected:
retrieving video data having the first quality for the fourth subset of the plurality of regions;
retrieving video data having the second quality for a fifth subset of the plurality of regions that spatially neighbor the fourth subset of the plurality of regions; and
retrieving video data having the third quality for a sixth subset of the plurality of regions that is outside the fourth subset and the fifth subset.

### Claim 7 (depends on 6)

The method of claim 6, wherein the fourth subset at least partially overlaps the second subset.

### Claim 8 (depends on 1)

The method of claim 1, wherein retrieving the video data comprises retrieving a left-eye view and a right-eye view for each of the regions.

### Claim 9 (depends on 8)

The method of claim 8, wherein for at least one of the regions, the left-eye view includes switch points at a regular interval, the right-eye view includes switch points at the regular interval, and the switch points of the right-eye view have different playback times than the switch points of the left-eye view.

### Claim 10 (depends on 9)

The method of claim 9, wherein the switch points of the right eye view occur half-way between the switch points of the left-eye view.

### Claim 11 (depends on 9)

The method of claim 9, further comprising, in response to determining that the focus of the user has changed, for at least one of the regions:
when an upcoming switch point for the left-eye view is earlier than an upcoming switch point for the right eye view, switching to a different set of video data for the left-eye view before switching to a different set of video data for the right-eye view; and
when the upcoming switch point for the right-eye view is earlier than the upcoming switch point for the left-eye view, switching to a different set of video data for the right-eye view before switching to a different set of video data for the left-eye view.

### Claim 12 (depends on 1)

The method of claim 1, wherein the display comprises six faces, each of the faces being divided into four regions for a total of twenty-four regions.

### Claim 13 (depends on 12)

The method of claim 12, wherein the first subset comprises four of the twenty-four regions, wherein the second subset comprises eight of the twenty-four regions, and wherein the third subset comprises eight of the twenty-four regions, the method further comprising retrieving video data having a fourth quality lower than the third quality for one or more remaining regions outside of the first subset, the second subset, and the third subset.

### Claim 14 (depends on 1)

The method of claim 1, wherein the first quality comprises 6k resolution, wherein the second quality comprises 4k resolution, and wherein the third quality comprises 1080p resolution.

### Claim 15 (depends on 14)

The method of claim 14, further comprising retrieving video data having 720p resolution for one or more remaining regions outside of the first subset, the second subset, and the third subset.

### Claim 16 (depends on 1)

The method of claim 1, further comprising receiving metadata indicating, for each of the regions, available qualities of video data for the region, wherein the available qualities correspond to at least one of spatial resolution, quantization parameters (QPs), peak signal to noise ratio (PSNR), or bitrate.

### Claim 17 (depends on 1)

The method of claim 1, further comprising receiving metadata indicating, for each of the regions, whether video data is available for only the region or whether video data is available for the region and one or more additional neighboring regions.

### Claim 18 (depends on 1)

The method of claim 1, further comprising receiving metadata indicating, for each of the regions, positions of switch points within the video data for the region.

### Claim 19 (depends on 1)

The method of claim 1, wherein retrieving the video data having the first quality for the first subset of the plurality of regions comprises:
when the visual focus of the user is directed to a center of a cube face of a cube map projection, retrieving a first mega-tile comprising video data for the cube face and portions of cube faces neighboring the cube face, wherein the first mega-tile has the first quality; and
when the visual focus of the user is directed to a vertex of the cube map projection, wherein three of the cube faces form the vertex, retrieving a second mega-tile comprising video data for regions of each of the three of the cube faces forming the vertex, wherein the second mega-tile has the first quality.

### Claim 20

A device for retrieving video data, the device comprising:
a display interface coupled to a display;
a memory configured to store video data; and
one or more processors configured to:
determine a plurality of regions of video data to be displayed via the display;
retrieve video data having a first quality for a first subset of the plurality of regions at which a visual focus of a user is directed;
retrieve video data having a second quality for a second subset of the plurality of regions that neighbor the first subset of the plurality of regions,
wherein the second quality is lower than the first quality; and
retrieve video data having a third quality for a third subset of the plurality of regions that is outside the first subset and the second subset, wherein the third quality is lower than the second quality.

### Claim 21 (depends on 20)

The device of claim 20, wherein the one or more processors are configured to:
retrieve video data having a first spatial resolution for the first subset of the plurality of regions,
retrieve video data having a second spatial resolution that is lower than the first spatial resolution for the second subset of the plurality of regions, wherein the video data for the second subset of the plurality of regions is coded independently of the video data having the first spatial resolution, and
retrieve video data having a third spatial resolution that is lower than the second spatial resolution for the third subset of the plurality of regions, wherein the video data for the third subset of the plurality of regions is coded independently of the video data having the first spatial resolution and the video data having the second spatial resolution.

### Claim 22 (depends on 20)

The device of claim 20, wherein the first subset of the plurality of regions comprises two or more regions, wherein video data is separately available for each of the two or more regions, and wherein the one or more processors are further configured to:
determine whether a single set of video data is available for each of the two or more regions; and
when the single set of video data is available for each of the two or more regions, retrieve the single set of video data without retrieving the video data that is separately available for each of the two or more regions.

### Claim 23 (depends on 20)

The device of claim 20, wherein the one or more processors are configured to retrieve a left-eye view and a right-eye view for each of the regions, wherein for at least one of the regions, the left-eye view includes switch points at a regular interval, the right-eye view includes switch points at the regular interval, and the switch points of the right-eye view have different playback times than the switch points of the left-eye view.

### Claim 24 (depends on 23)

The device of claim 23, wherein the one or more processors are further configured to, in response to determining that the focus of the user has changed, for at least one of the regions:
when an upcoming switch point for the left-eye view is earlier than an upcoming switch point for the right eye view, switch to a different set of video data for the left-eye view before switching to a different set of video data for the right-eye view; and
when the upcoming switch point for the right-eye view is earlier than the upcoming switch point for the left-eye view, switch to a different set of video data for the right-eye view before switching to a different set of video data for the left-eye view.

### Claim 25 (depends on 20)

The device of claim 20, wherein the device comprises at least one of:
an integrated circuit;
a microprocessor; or
a wireless communication device.

### Claim 26

A device for retrieving video data, the device comprising:
means for determining a plurality of regions of video data to be displayed via a display;
means for retrieving video data having a first quality for a first subset of the plurality of regions at which a visual focus of a user is directed;
means for retrieving video data having a second quality for a second subset of the plurality of regions that neighbor the first subset of the plurality of regions, wherein the second quality is lower than the first quality; and
means for retrieving video data having a third quality for a third subset of the plurality of regions that is outside the first subset and the second subset, wherein the third quality is lower than the second quality.

### Claim 27

A computer-readable storage medium having stored thereon instructions that, when executed, cause a processor to:
determine a plurality of regions of video data to be displayed via a display;
retrieve video data having a first quality for a first subset of the plurality of regions at which a visual focus of a user is directed;
retrieve video data having a second quality for a second subset of the plurality of regions that neighbor the first subset of the plurality of regions, wherein the second quality is lower than the first quality; and
retrieve video data having a third quality for a third subset of the plurality of regions that is outside the first subset and the second subset, wherein the third quality is lower than the second quality.

### Claim 28 (depends on 27)

The computer-readable storage medium of claim 27,
wherein the instructions that cause the processor to retrieve the video data having the first quality comprise instructions that cause the processor to retrieve video data having a first spatial resolution for the first subset of the plurality of regions;
wherein the instructions that cause the processor to retrieve the video data having the second quality comprise instructions that cause the processor to retrieve video data having a second spatial resolution that is lower than the first spatial resolution for the second subset of the plurality of regions, wherein the video data for the second subset of the plurality of regions is coded independently of the video data having the first spatial resolution; and
wherein the instructions that cause the processor to retrieve the video data comprise instructions that cause the processor to retrieve video data having a third spatial resolution that is lower than the second spatial resolution for the third subset of the plurality of regions, wherein the video data for the third subset of the plurality of regions is coded independently of the video data having the first spatial resolution and the video data having the second spatial resolution.

### Claim 29 (depends on 27)

The computer-readable storage medium of claim 27, further comprising instructions that cause the processor to retrieve a left-eye view and a right-eye view for each of the regions, wherein for at least one of the regions, the left-eye view includes switch points at a regular interval, the right-eye view includes switch points at the regular interval, and the switch points of the right-eye view have different playback times than the switch points of the left-eye view.

### Claim 30 (depends on 29)

The computer-readable storage medium of claim 29, further comprising instructions that cause the processor to, in response to determining that the focus of the user has changed, for at least one of the regions:
when an upcoming switch point for the left-eye view is earlier than an upcoming switch point for the right eye view, switch to a different set of video data for the left-eye view before switching to a different set of video data for the right-eye view; and
when the upcoming switch point for the right-eye view is earlier than the upcoming switch point for the left-eye view, switch to a different set of video data for the right-eye view before switching to a different set of video data for the left-eye view.
