# US 2020/0404241 A1 · Processing system for streaming volumetric video to a client device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404241 A1
- Document ID: US20200404241A1
- Status: Published application
- Kind: A1
- Application number: 16/446,815
- Filed: 2019-06-20
- Published: 2020-12-24
- Earliest claimed: 2019-06-20
- Assignees: AT&T Intellectual Property I, L.P.
- Applicants: AT&T Intellectual Property I, L.P.
- Inventors: Bo Han; Jackson Jarrell Pair; Tan Xu
- Web page: https://patentgrep.com/patent/US20200404241A1

## CPC

- G06T 9/001: {Model-based coding, e.g. wire frame}
- H04N 13/194: Transmission of image signals
- G06T 1/60: Memory management
- G06T 9/002: {using neural networks}
- G06T 9/40: Tree coding, e.g. quadtree, octree
- H04N 7/0117: {involving conversion of the spatial resolution of the incoming video signal (for graphics images G09G2340/0407)}
- H04N 13/111: Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A network processing system obtains a viewport of a client device for volumetric video and a two-dimensional (2D) subframe of a frame of volumetric video is obtained associated with the viewport. Viewports may be obtained from the client device or be predicted. 2D subframes and reduced resolution versions of frames can be transmitted to the client device. A client device may request volumetric video from the network processing system and provides a viewport to the network processing system. The client device may obtain from the network processing system reduced resolution versions of volumetric video frames and 2D subframes in accordance with the viewport. The client device may determine whether a current viewport matches the viewport associated with the obtained 2D subframe and provides a display based on either that subframe (upon a match) or a 2D perspective of the reduced resolution frame associated with the current viewport (if no match).

## Claims

### Claim 1

A method comprising:
obtaining, by a processing system including at least one processor, a viewport of a client device for a volumetric video;
obtaining, by the processing system, a two-dimensional subframe of a frame of the volumetric video, the two-dimensional subframe associated with the viewport of the client device; and
transmitting, by the processing system to the client device, the two-dimensional subframe and a three-dimensional reduced resolution version of the frame of the volumetric video.

### Claim 2

(canceled)

### Claim 3 (depends on 1)

The method of claim 1, wherein the viewport comprises a predicted viewport based upon a plurality of prior viewports.

### Claim 4 (depends on 3)

The method of claim 3, wherein the predicted viewport is predicted in accordance with a machine learning model.

### Claim 5 (depends on 1)

The method of claim 1, wherein the viewport is obtained from the client device.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
obtaining the frame of the volumetric video.

### Claim 7 (depends on 6)

The method of claim 6, wherein the frame of the volumetric video is obtained as part of at least a portion of the volumetric video comprising a plurality of encoded frames, wherein the method further comprises:
decoding the frame of the volumetric video from the plurality of encoded frames.

### Claim 8 (depends on 6)

The method of claim 6, further comprising:
generating the three-dimensional reduced resolution version of the frame.

### Claim 9 (depends on 8)

The method of claim 8, wherein the generating the three-dimensional reduced resolution version of the frame comprises:
performing an interframe encoding between the three-dimensional reduced resolution version of the frame and at least one additional three-dimensional reduced resolution version of at least one additional frame of the volumetric video.

### Claim 10 (depends on 8)

The method of claim 8, wherein the generating the three-dimensional reduced resolution version of the frame comprises applying an octree-based compression.

### Claim 11 (depends on 1)

The method of claim 1, wherein the three-dimensional reduced resolution version of the frame is selected based upon a throughput between the client device and the processing system.

### Claim 12 (depends on 1)

The method of claim 1, wherein the frame of the volumetric video comprises:
a point cloud; or
a three-dimensional mesh.

### Claim 13 (depends on 1)

The method of claim 1, wherein the obtaining the two-dimensional subframe comprises:
generating the two-dimensional subframe from the frame of the volumetric video.

### Claim 14 (depends on 13)

The method of claim 13, where the generating the two-dimensional subframe comprises generating a plurality of two-dimensional subframes, wherein each of the plurality of two-dimensional subframes is generated from a respective time sequential frame of the volumetric video, the method further comprising:
applying an interframe encoding to the plurality of two-dimensional subframes.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
caching, by the processing system, at least one of:
the two-dimensional subframe; or
the three-dimensional reduced resolution version of the frame.

### Claim 16 (depends on 1)

The method of claim 1, wherein the two-dimensional subframe is obtained from a cache of the processing system.

### Claim 17 (depends on 1)

The method of claim 1, wherein the viewport comprises a position and an orientation.

### Claim 18 (depends on 17)

The method of claim 17, wherein the orientation comprises a yaw, a pitch, and a roll.

### Claim 19

A device comprising:
a processor; and
a computer-readable medium storing instructions which, when executed by the processor, cause the processor to perform operations, the operations comprising:
obtaining a viewport of a client device for a volumetric video;
obtaining a two-dimensional subframe of a frame of the volumetric video, the two-dimensional subframe associated with the viewport of the client device; and
transmitting, to the client device, the two-dimensional subframe and a three-dimensional reduced resolution version of the frame of the volumetric video.

### Claim 20

A non-transitory computer-readable medium storing instructions which, when executed by a processing system including at least one processor, cause the processing system to perform operations, the operations comprising:
obtaining a viewport of a client device for a volumetric video;
obtaining a two-dimensional subframe of a frame of the volumetric video, the two-dimensional subframe associated with the viewport of the client device; and
transmitting, to the client device, the two-dimensional subframe and a three-dimensional reduced resolution version of the frame of the volumetric video.

### Claim 21 (depends on 8)

The method of claim 8, wherein the generating the three-dimensional reduced resolution version of the frame comprises applying compression.
