# US 2018/0025530 A1 · System and method for geometric warping correction in projection mapping

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0025530 A1
- Document ID: US20180025530A1
- Status: Published application
- Kind: A1
- Application number: 15/215,668
- Filed: 2016-07-21
- Published: 2018-01-25
- Earliest claimed: 2016-07-21
- Assignees: Christie Digital Systems USA, Inc.
- Applicants: Christie Digital Systems USA, Inc.
- Inventors: Roy ANTHONY; Kevin MOULE; Derek SCOTT; Nick WASILKA; Maxwell ELENDT
- Web page: https://patentgrep.com/patent/US20180025530A1

## CPC

- G06T 15/10: Geometric effects
- G06T 3/08: Projecting images onto non-planar surfaces, e.g. geodetic screens
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- H04N 9/3147: {Multi-projection systems (video walls G06F3/1446)}
- H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- H04N 9/3194: {including sensor feedback}
- G06T 2210/36: Level of detail
- G06T 2219/2021: Shape modification
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system and method for geometric warping correction in projection mapping is provided. A lower resolution mesh is applied to A mesh model, at least in a region of the mesh model misaligned with a corresponding region of a real-world object. One or more points of the lower resolution mesh are moved. In response, one or more corresponding points of the mesh model are moved to increase alignment between the region of the mesh model and the corresponding region of the real-world object. An updated mesh model is stored in a memory. And one or more projectors are controlled to projection map images corresponding to the updated mesh model onto the real-world object.

## Claims

### Claim 1

A system comprising:
one or more projectors configured to projection map onto a real-world object;
a memory storing a mesh model of the real-world object; and,
a controller configured to:
apply a lower resolution mesh to the mesh model, at least in a region of the mesh model misaligned with a corresponding region of the real-world object, the lower resolution mesh having one or more of a lower resolution and a lower point density than the mesh model;
move one or more low-res points of the lower resolution mesh; and, in response, move one or more corresponding high-res points of the mesh model, including any high-res points located between the one or more low-res points, to increase alignment between the region of the mesh model and the corresponding region of the real-world object;
store an updated mesh model in the memory based on the one or more corresponding high-res points of the mesh model that moved with the one or more low-res points of the lower resolution mesh; and,
control the one or more projectors to projection map images corresponding to the updated mesh model onto the real-world object.

### Claim 2 (depends on 1)

The system of claim 1, wherein the lower resolution mesh comprises one or more of: a vertex cage; a vertex clustering simplification of the mesh model; an incremental decimation of the mesh model; and, a subset of the high-res points of the mesh model.

### Claim 3 (depends on 1)

The system of claim 1, further comprising an input device, wherein the controller is further configured to determine the regions of the mesh model misaligned with the corresponding regions of the real-world object by: receiving input from the input device identifying the regions.

### Claim 4 (depends on 1)

The system of claim 1, wherein the lower resolution mesh is generated manually via input selecting sets of high-res points of the mesh model to be affected by a single newly created low res-point at a center of gravity of selected high-res points.

### Claim 5 (depends on 1)

The system of claim 1, wherein the controller is further configured to determine the region of the mesh model misaligned with the corresponding region of the real-world object by: comparing one or more of the mesh model and the lower resolution mesh with a point cloud representing the real-world object.

### Claim 6 (depends on 1)

The system of claim 1, wherein the controller is further configured to determine the region of the mesh model misaligned with the corresponding region of the real-world object by: comparing one or more of the mesh model and the lower resolution mesh with a point cloud representing the real-world object using one or more of point-comparison techniques, feature matching techniques, key-feature-related techniques and normal-related techniques.

### Claim 7 (depends on 1)

The system of claim 1, wherein the controller is further configured to automatically determine the region of the mesh model misaligned with the corresponding region of the real-world object by one or more of: calculating a respective distance of each of the one or more low-res points of the lower resolution mesh to a respective closest point in a point cloud representing the real-world object; and calculating a respective distance of each of the one or more high-res points of the mesh model to a respective closest point in a point cloud representing the real-world object.

### Claim 8 (depends on 1)

The system of claim 1, wherein the controller is further configured to automatically move one or more low-res points of the lower resolution mesh to an intersection of a respective lines determined from respective normals of respective vertexes of the lower resolution mesh and a mesh generated from a point cloud representing the real-world object until an alignment occurs between the region of the mesh model and a respective corresponding region of the point cloud.

### Claim 9 (depends on 1)

The system of claim 1, wherein the controller is further configured to move one or more low-res points of the lower resolution mesh until an alignment occurs between the region of the mesh model and a respective corresponding region of a point cloud representing the real-world object.

### Claim 10 (depends on 1)

The system of claim 1, wherein the controller is further configured to control the one or more projectors to projection map respective images corresponding to one or more intermediate updated mesh models onto the real-world object while the one or more low-res points of the lower resolution mesh are being moved.

### Claim 11

A method comprising:
at a system comprising: one or more projectors configured to projection map onto a real-world object; a memory storing a mesh model of the real-world object; and, a controller, applying, using the controller, a lower resolution mesh to the mesh model, at least in a region of the mesh model misaligned with a corresponding region of the real-world object, the lower resolution mesh having one or more of a lower resolution and a lower point density than the mesh model;
moving, using the controller, one or more low-res points of the lower resolution mesh; and, in response, move one or more corresponding high-res points of the mesh model, including any high-res points located between the one or more low-res points, to increase alignment between the region of the mesh model and the corresponding region of the real-world object;
storing, using the controller, an updated mesh model in the memory based on the one or more corresponding high-res points of the mesh model that moved with the one or more low-res points of the lower resolution mesh; and,
controlling, using the controller, the one or more projectors to projection map images corresponding to the updated mesh model onto the real-world object.

### Claim 12 (depends on 11)

The method of claim 11, wherein the lower resolution mesh comprises one or more of: a vertex cage; a vertex clustering simplification of the mesh model; an incremental decimation of the mesh model; and, a subset of the high-res points of the mesh model.

### Claim 13 (depends on 11)

The method of claim 11, wherein the system further comprises an input device, and the method further comprises determining the regions of the mesh model misaligned with the corresponding regions of the real-world object by: receiving input from the input device identifying the regions.

### Claim 14 (depends on 11)

The method of claim 11, further comprising generating the lower resolution mesh manually via input selecting sets of high-res points of the mesh model to be affected by a single newly created low res-point at a center of gravity of selected high-res points.

### Claim 15 (depends on 11)

The method of claim 11, further comprising determining the region of the mesh model misaligned with the corresponding region of the real-world object by: comparing one or more of the mesh model and the lower resolution mesh with a point cloud representing the real-world object.

### Claim 16 (depends on 11)

The method of claim 11, further comprising determining the region of the mesh model misaligned with the corresponding region of the real-world object by one or more of: comparing one or more of the mesh model and the lower resolution mesh with a point cloud representing the real-world object using one or more of point-comparison techniques, feature matching techniques, key-feature-related techniques and normal-related techniques.

### Claim 17 (depends on 11)

The method of claim 11, further comprising automatically determining the region of the mesh model misaligned with the corresponding region of the real-world object by one or more of: calculating a respective distance of each of the one or more low-res points of the lower resolution mesh to a respective closest point in a point cloud representing the real-world object; and calculating a respective distance of each of the one or more high-res points of the mesh model to a respective closest point in a point cloud representing the real-world object.

### Claim 18 (depends on 11)

The method of claim 11, further comprising automatically moving one or more low-res points of the lower resolution mesh to an intersection of a respective lines determined from respective normals of respective vertexes of the lower resolution mesh and a mesh generated from a point cloud representing the real-world object until an alignment occurs between the region of the mesh model and a respective corresponding region of the point cloud.

### Claim 19 (depends on 11)

The method of claim 11, further comprising moving one or more low-res points of the lower resolution mesh until an alignment occurs between the region of the mesh model and a respective corresponding region of a point cloud representing the real-world object.

### Claim 20 (depends on 11)

The method of claim 11, further comprising controlling the one or more projectors to projection map respective images corresponding to one or more intermediate updated mesh models onto the real-world object while the one or more low-res points of the lower resolution mesh are being moved.
