# US 2024/0193747 A1 · System and Method for Point Cloud Generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0193747 A1
- Document ID: US20240193747A1
- Status: Published application
- Kind: A1
- Application number: 18/285,462
- Filed: 2022-03-02
- Published: 2024-06-13
- Earliest claimed: 2021-04-07
- Assignees: Sony Interactive Entertainment Europe Limited; The University of York
- Applicants: Sony Interactive Entertainment Europe Limited; University of York
- Inventors: Timothy Bradley; Marina Villanueva Barreiro; Oliver Hume; Ryan Spick; James Walker
- Web page: https://patentgrep.com/patent/US20240193747A1

## CPC

- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30168: Image quality inspection
- G06T 2210/56: Particle system, point based geometry or rendering
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method of point cloud generation comprises generating, by a processor, a first point cloud comprising a first set of generated data points; calculating, by the processor, a loss function representative of the quality of the first point cloud, the loss function comprising a smoothness term obtained by constructing vectors between neighbouring data points in the point cloud, performing vector multiplication between neighbouring pairs of vectors, and outputting a smoothness term based on the result of the vector multiplication; and generating, by the processor, a second point cloud comprising a second set of generated data points based at least in part on the calculated loss function of the first point cloud.

## Claims

### Claim 1

A method of point cloud generation comprising:
generating, by a processor, a first point cloud comprising a first set of generated data points;
calculating, by the processor, a loss function representative of the quality of the first point cloud, the loss function comprising a smoothness term obtained by constructing vectors between neighbouring data points in the point cloud, performing vector multiplication between neighbouring pairs of vectors, and outputting a smoothness term based on the result of the vector multiplication; and
generating, by the processor, a second point cloud comprising a second set of generated data points based at least in part on the calculated loss function of the first point cloud.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the smoothness term for the point cloud is obtained by taking an average value of a dot product between neighbouring vectors in the point cloud.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the smoothness term, S, is calculated as:
S
=
1
-
|
∑
i
=
0
n
-
1
⁢
v
i
·
v
i
+
1
n
|
where vi and vi+1 are neighbouring pairs of vectors each constructed between neighbouring data points in the point cloud, and n is a total number of vectors constructed in the data point set.

### Claim 4 (depends on 3)

The method according to claim 3, in which contributions to the smoothness term from respective neighbouring pairs of vectors is responsive to a corresponding local smoothness of a reference model.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the method comprises an initial step of receiving, from a user, an input vicinity control, and wherein the vector multiplication is done between vectors within a spatial vicinity corresponding to the input vicinity control.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the first and second point clouds are generated by a machine learning system that is updated responsive to the loss function between generation of the first and second point clouds.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the step of generating a first point cloud comprises inputting a noise vector to a generator of a generative adversarial network, GAN, trained to generate a set of generated data points and output the first point cloud.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the step of calculating a loss function comprises the steps of:
inputting the first point cloud to a discriminator of the GAN, trained to analyse the point cloud and output a binary cross entropy, BCE, term;
calculating the smoothness term; and
combining a pre-determined ratio of the BCE loss term and the smoothness term to form the loss function.

### Claim 9 (depends on 1)

The method according to claim 1, wherein the step of generating a second point cloud comprises comparing the calculated smoothness term of the first point cloud to a threshold smoothness value and generating the second point cloud based at least in part on the comparison.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the threshold smoothness value is determined by calculating a smoothness term of one or more training point cloud data used to train the GAN.

### Claim 11

A computer program comprising computer-implemented instructions that, when run on a computer, cause the computer to implement a method of point cloud generation comprising:
generating, by a processor, a first point cloud comprising a first set of generated data points;
calculating, by the processor, a loss function representative of the quality of the first point cloud, the loss function comprising a smoothness term obtained by constructing vectors between neighboring data points in the point cloud, performing vector multiplication between neighboring pairs of vectors, and outputting a smoothness term based on the result of the vector multiplication; and
generating, by the processor, a second point cloud comprising a second set of generated data points based at least in part on the calculated loss function of the first point cloud.

### Claim 12

A system for point cloud generation comprising one or more processors comprising:
a point cloud generation unit, configured to generate a first point cloud comprising a first set of generated data points;
a loss function determining unit, configured to calculate a loss function representative of the quality of the first point cloud, the loss function comprising a smoothness term obtained by constructing vectors between neighbouring data points in the point cloud, perform vector multiplication between neighbouring pairs of vectors and output a smoothness term based on the result of vector multiplication, wherein
the point cloud generation unit is further configured to generate a second point cloud comprising a second set of generated data points based at least in part on the calculated loss function of the first point cloud.

### Claim 13 (depends on 12)

A system according to claim 12, further comprising a memory, wherein the memory includes at least one generator of a GAN and at least one discriminator of a GAN, wherein the generator and the discriminator are respectively configured to operate the point cloud generation unit and the loss function determining unit.

### Claim 14 (depends on 13)

A system according to claim 13, wherein the point cloud generation unit is configured to store generated point clouds on the memory and the loss function determining unit is arranged to access the memory and retrieve generated point clouds from the memory.

### Claim 15 (depends on 12)

A system for point cloud generation comprising:
a point cloud generation processor adapted to generate point clouds using a machine learning system previously trained using the system of claim 12.
