# US 2024/0202979 A1 · Graph signal frequency-domain motion compensation for point cloud attributes coding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202979 A1
- Document ID: US20240202979A1
- Status: Published application
- Kind: A1
- Application number: 18/084,074
- Filed: 2022-12-19
- Published: 2024-06-20
- Earliest claimed: 2022-12-19
- Assignees: Adeia Guides Inc.
- Applicants: Rovi Guides, Inc.
- Inventors: Zhu Li; Tao Chen
- Web page: https://patentgrep.com/patent/US20240202979A1

## CPC

- G06T 9/001: {Model-based coding, e.g. wire frame}
- G06T 9/40: Tree coding, e.g. quadtree, octree
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods are disclosed herein for temporally predictive coding of three-dimensional (3D) dynamic point cloud attributes. A first frame and a second frame of point cloud data are accessed. The point cloud data points include 3D spatial coordinates and one or more graphic attributes. A block tree data structure comprising a plurality of blocks is generated based on a tree partitioning of the second frame of point cloud data. Matching block pairs between the first frame and the second frame are identified from the plurality of blocks based on block-wise searching. Frequency-domain projections are generated for each matching block pair via a graph Fourier transform (GFT) algorithm. A bitstream of motion-compensated residuals is generated based on differences in the frequency-domain projections for each matching block pair.

## Claims

### Claim 1

A computer-implemented method for temporally predictive coding of three-dimensional (3D) dynamic point cloud data, the method comprising:
accessing a first frame of 3D point cloud data and a second frame of 3D point cloud data, each 3D point cloud data point comprising 3D spatial coordinates and one or more graphic attributes;
generating a block tree data structure based on a tree partitioning of the second frame of 3D point cloud data, the block tree data structure comprising a plurality of blocks;
identifying, from the plurality of blocks, a plurality of matching block pairs between the first frame and the second frame based on block-wise searching of the block tree data structure, wherein each matching block pair comprises a second block of the second frame and a first block of the first frame that matches the second block;
for each matching block pair of the plurality of matching block pairs:
generating frequency-domain representations of a respective first block of the first frame and a respective second block of the second frame via a graph Fourier transform (GFT) algorithm; and
determining differences in the frequency-domain representations of the respective first block of the first frame and the respective second block of the second frame; and
generating a bitstream of motion-compensated residuals based on the determined differences in the frequency-domain representations of the plurality of matching block pairs.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the differences in the frequency-domain representations of the respective first block of the first frame and the respective second block of the second frame comprises:
determining a residual between the frequency-domain representations, wherein the residual represents the differences in the frequency-domain representations; and
quantizing the residual.

### Claim 3 (depends on 2)

The method of claim 2, wherein generating a bitstream of motion-compensated residuals further comprises generating the bitstream based on entropy coding the quantized residual.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
for each block of the plurality of blocks of the block tree data structure:
comparing a first encoding cost of the residual and a second encoding cost of a respective block of the second frame; and
in response to determining that the second encoding cost is less than the first encoding cost of the respective block, switching from an inter coding mode to an intra coding mode for the respective block.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the block tree data structure comprises partitioning the 3D point cloud data via a binary space partitioning algorithm.

### Claim 6 (depends on 5)

The method of claim 5, wherein the binary space partitioning algorithm is a k-dimensional (k-d) tree algorithm.

### Claim 7 (depends on 1)

The method of claim 1, wherein identifying the plurality of matching block pairs comprises determining block-wise motion prediction corresponding to the plurality of matching block pairs via an iterative closest point (ICP) algorithm.

### Claim 8 (depends on 1)

The method of claim 1, wherein the 3D point cloud data comprises non-uniformly sampled data points.

### Claim 9 (depends on 1)

The method of claim 1, wherein the one or more graphic attributes comprise one or more of additive color components, subtractive color components, chrominance components, or luminance components.

### Claim 10 (depends on 1)

The method of claim 1, wherein the bitstream of motion-compensated residuals is generated based on differences in the one or more graphic attributes of the plurality of matching block pairs.

### Claim 11

A system for temporally predictive coding of three-dimensional (3D) dynamic point cloud data, the system comprising:
communication circuitry configured to transmit and retrieve content; and
control circuitry configured to:
access, via the communication circuitry, a first frame of 3D point cloud data and a second frame of 3D point cloud data, each 3D point cloud data point comprising 3D spatial coordinates and one or more graphic attributes;
generate a block tree data structure based on a tree partitioning of the second frame of 3D point cloud data, the block tree data structure comprising a plurality of blocks;
identify, from the plurality of blocks, a plurality of matching block pairs between the first frame and the second frame based on block-wise searching of the block tree data structure, wherein each matching block pair comprises a second block of the second frame and a first block of the first frame that matches the second block;
for each matching block pair of the plurality of matching block pairs:
generate frequency-domain representations of a respective first block of the first frame and a respective second block of the second frame via a graph Fourier transform (GFT) algorithm; and
determine differences in the frequency-domain representations of the respective first block of the first frame and the respective second block of the second frame; and
generate a bitstream of motion-compensated residuals based on the determined differences in the frequency-domain representations of the plurality of matching block pairs.

### Claim 12 (depends on 11)

The system of claim 11, wherein the control circuitry, when determining the differences in the frequency-domain representations, is configured to:
determine a residual between the frequency-domain representations, wherein the residual represents the differences in the frequency-domain representations; and
quantize the residual.

### Claim 13 (depends on 12)

The system of claim 12, wherein the control circuitry is further configured to generate the bitstream based on entropy coding the quantized residual.

### Claim 14 (depends on 12)

The system of claim 12, wherein the control circuitry is further configured to:
for each block of the plurality of blocks of the block tree data structure:
compare a first encoding cost of the residual and a second encoding cost of a respective block of the second frame; and
in response to determining that the second encoding cost is less than the first encoding cost of the respective block, switch from an inter coding mode to an intra coding mode for the respective block.

### Claim 15 (depends on 11)

The system of claim 11, wherein the control circuitry, when generating the block tree data structure, is configured to partition the 3D point cloud data via a binary space partitioning algorithm.

### Claim 16 (depends on 15)

The system of claim 15, wherein the binary space partitioning algorithm is a k-dimensional (k-d) tree algorithm.

### Claim 17 (depends on 11)

The system of claim 11, wherein the control circuitry, when identifying the plurality of matching block pairs, is configured to determine block-wise motion prediction corresponding to the plurality of matching block pairs via an iterative closest point (ICP) algorithm.

### Claim 18 (depends on 11)

The system of claim 11, wherein the 3D point cloud data comprises non-uniformly sampled data points.

### Claim 19 (depends on 11)

The system of claim 11, wherein the one or more graphic attributes comprise one or more of additive color components, subtractive color components, chrominance components, or luminance components.

### Claim 20 (depends on 11)

The system of claim 11, wherein the control circuitry is configured to generate the bitstream of motion-compensated residuals based on differences in the one or more graphic attributes of the plurality of matching block pairs.

### Claim at position 21

21-30. (canceled)
