# US 2022/0191442 A1 · Camera-assisted projection optics distortion characterization and correction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0191442 A1
- Document ID: US20220191442A1
- Status: Published application
- Kind: A1
- Application number: 17/118,327
- Filed: 2020-12-10
- Published: 2022-06-16
- Earliest claimed: 2020-12-10
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Jaime Rene DE LA CRUZ VAZQUEZ; Jeffrey Matthew KEMPF
- Web page: https://patentgrep.com/patent/US20220191442A1

## CPC

- H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- G06T 3/18: Image warping, e.g. rearranging pixels individually
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/80: Geometric correction
- H04N 9/3141: {Constructional details thereof (details not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- H04N 9/3194: {including sensor feedback}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/10152: Varying illumination
- G06T 2207/30208: Marker matrix
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method includes projecting an image onto a projection surface through a projection lens of a projector, where the image comprises a fiducial marker. The method also includes capturing a point cloud of the fiducial marker with a camera, and generating a distortion map of projection lens distortion based at least in part on the point cloud. The method also includes generating a correction map for the projection lens, and applying the correction map to a video signal input to the projector.

## Claims

### Claim 1

A method, comprising:
projecting a first structured light pattern by a projection lens of a projector while the projector and a camera are in a first pose;
capturing, by the camera, a first image of the first projected structured light pattern;
producing, by at least one processor, a first point-level representation of projected pixels based on the first image;
projecting a second structured light pattern onto the projection surface by the projection lens while the projector and the camera are in a second pose;
capturing, by the camera, a second image of the second projected structure light pattern;
producing, by the at least one processor, a second point-level representation of the projected pixels based on the second image;
generating a distortion map of projection lens distortion based on the first image and the second image; and
generating a correction based on the distortion map.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the correction map is performed using a homography transformation.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
applying the correction map to a projection image to produce a corrected image; and
projecting the corrected image.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
convolving a transient warping transformation and the correction map to create a convolution correction map;
applying the correction map to a projection image to produce a corrected image; and
projecting the corrected image.

### Claim 5 (depends on 1)

The method of claim 1, wherein the correction map is an inverse geometric transformation of the distortion map.

### Claim 6 (depends on 1)

The method of claim 1, further comprising cancelling at least a portion of distortion caused by the projection lens using a pixel displacement profile of the correction map.

### Claim 7 (depends on 1)

The method of claim 1, wherein the distortion map characterizes distortion as a pixel displacement with respect to the first image and the second image.

### Claim 8

(canceled)

### Claim 9 (depends on 1)

The method of claim 1, wherein the first structured light pattern comprises discrete structured light elements covering the first image.

### Claim 10 (depends on 1)

The method of claim 1, wherein the projector and the camera are mechanically attached to one another, their orientations relative to one another being fixed.

### Claim 11

A system, comprising:
a projector comprising projection optics; and
a controller coupled to the projector, the controller configured to:
obtain a point cloud of a fiducial marker;
generate a distortion map of distortion based at on the point cloud;
generate a correction map for the projection optics;
generate a transient warping map for the system;
convolve the correction map with the transient warping map to produce a convolution correction map;
apply the convolution correction map to a projection image to produce a corrected image; and
instruct the projector to project the corrected image, wherein the projector is configured to project the corrected image.

### Claim 12 (depends on 11)

The system of claim 11, wherein the correction map includes a pixel displacement profile that is configured to cancel distortion caused by the projection optics.

### Claim 13 (depends on 11)

The system of claim 11, wherein obtaining the point cloud includes projecting a structured light pattern onto a projection surface.

### Claim 14 (depends on 11)

The system of claim 11, wherein the projection optics and a camera are mechanically attached to one another, their orientations relative to one another being fixed.

### Claim 15 (depends on 11)

The system of claim 11, wherein the controller is further configured to generate the correction map using a homography transformation.

### Claim 16 (depends on 11)

The system of claim 11, wherein the correction map is an inverse geometric transformation of the distortion map.

### Claim 17

A system, comprising:
a projector comprising projection optics, the projector configured to:
project a first structured light pattern while the projector and a camera are in a first pose; and
project a second structured light pattern while the projector and the camera are in a second pose;
the camera configured to:
capture a first image of the first projected structured light pattern while the projector and the camera are in the first pose; and
capture a second image of the second projected structured light pattern while the projector and the camera are in the second pose; and
a controller coupled to the projector and to the camera, the controller configured to:
produce a first point-level representation of projected pixels based on the first image of the first structured light pattern;
produce a second point-level representation of projected pixels based on the second image of the second structured light pattern;
generate a distortion map based on the first point-level representation and the second point-level representation; and
generate a correction map based on the distortion map.

### Claim 18 (depends on 17)

The system of claim 17, wherein the controller is further configured to generate the correction map using a homography transformation.

### Claim 19 (depends on 17)

The system of claim 17, wherein the correction map is an inverse geometric transformation of the distortion map.

### Claim 20

(canceled)

### Claim 21 (depends on 17)

The system of claim 17, the controller further configured to produce a corrected image based on the correction map, and the projector further configured to project the corrected image.

### Claim 22 (depends on 11)

The projector of claim 11, the projector further configured to project the corrected image.
