# US 11,115,633 B2 · Method and system for projector calibration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,115,633 B2 - Document ID: US11115633B2 - Status: Granted - Kind: B2 - Application number: 16/059,882 - Filed: 2018-08-09 - Published: 2018-12-27 - Granted: 2021-09-07 - Earliest claimed: 2014-11-06 - Assignees: Disney Enterprises, Inc. - Applicants: Disney Enterprises, Inc. - Inventors: Anselm Grundhöfer - Web page: https://patentgrep.com/patent/US11115633B2 ## CPC - H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)} - G01B 11/2504: {Calibration devices} - G01B 11/2513: {with several lines being projected in more than one direction, e.g. grids, patterns} - G06T 3/18: Image warping, e.g. rearranging pixels individually - H04N 9/3194: {including sensor feedback} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present disclosure relates to a method for calibrating a projector. The method includes projecting a test pattern onto a scene or an object within the scene and capturing the test pattern using a camera. Once the test pattern image has been captured by the camera, the method further includes estimating by a processing element a perspective projection matrix, warping the estimated projection matrix based on a non-linear distortion function, and modifying the projector to project light based on the distortion function. The present disclosure also relates to a presentation or projection system including two types of projectors for projecting an output presentation having a second image overlaid on a first image. ## Claims ### Claim 1 A method of calibrating a projector comprising: receiving by a processing element a first calibration image of a first calibration pattern projected onto a projection surface; generating a perspective projection matrix by the processing element by modeling the projector as a pinhole device with a pinhole approximation to provide a first calibration approximation based on the first calibration image and the first calibration pattern; and determining by the processing element a two-dimensional non-linear mapping function to correct inaccuracies in the first calibration approximation resulting from the pinhole approximation by analyzing the first calibration image to compensate for distortion characteristics inherent to the projector. ### Claim 2 (depends on 1) The method of claim 1, wherein the first calibration pattern comprises a plurality of pattern elements. ### Claim 3 (depends on 2) The method of claim 2, further comprising detecting by the processing element the plurality of pattern elements using an image analysis algorithm and determining by the processing element a three-dimensional position of the plurality of pattern elements. ### Claim 4 (depends on 2) The method of claim 2, further comprising determining by the processing element a transformation amount for each of the pluralities of pattern elements. ### Claim 5 (depends on 1) The method of claim 1, wherein the non-linear mapping function is determined based on the perspective projection matrix and the first calibration image. ### Claim 6 (depends on 5) The method of claim 5, wherein the non-linear mapping function is determined by comparing by the processing element the first calibration image and the perspective projection matrix. ### Claim 7 (depends on 1) The method of claim 1, further comprising generating by the processing element a calibrated image by transforming the first calibration image using the non-linear mapping function. ### Claim 8 (depends on 1) The method of claim 1, further comprising creating by the processing element a look up table using the non-linear mapping function. ### Claim 9 (depends on 8) The method of claim 8, further comprising interpolating by the processing element values within the look up table to increase the number of values. ### Claim 10 (depends on 1) The method of claim 1, further comprising: receiving by the processing element a second calibration image of a second calibration pattern projected onto a projection surface, wherein the second calibration pattern is different than the first calibration pattern. ### Claim 11 (depends on 10) The method of claim 10, wherein generation of the perspective projection matrix further comprises analyzing the second calibration image and the second calibration pattern. ### Claim 12 (depends on 1) The method of claim 1, wherein the two-dimensional non-linear mapping function results in a second calibration approximation that is more accurate than the first calibration approximation. ### Claim 13 A system for calibrating a laser projector with non-uniform distortion characteristics comprising: a laser projector, wherein the laser projector projects a first calibration pattern onto a three-dimensional projection surface; a camera, wherein the camera captures a first calibration image of the first calibration pattern projected onto the three-dimensional projection surface; and a computing device, in electrical communication with the camera and having at least one processing element, performs the following operations: generates a perspective projection matrix by modeling the laser projector as a pinhole device with a pinhole approximation to provide a first calibration approximation based on the first calibration image; and determines a distortion map for the laser projector that maps two-dimensional positions on a virtual image plane of the laser projector to three-dimensional positions on the three-dimensional projection surface by comparing the first calibration approximation to the first calibration image and the first calibration pattern to correct inaccuracies in the first calibration approximation resulting from the pinhole approximation, wherein the distortion map determines distortion inherent to the laser projector. ### Claim 14 (depends on 13) The system of claim 13, wherein the computing device further analyzes the first calibration image with an image analysis algorithm to determine a plurality of pattern elements of the first calibration pattern within the first calibration image. ### Claim 15 (depends on 14) The system of claim 14, wherein the first calibration approximation is based on the analysis of the first calibration image. ### Claim 16 (depends on 15) The system of claim 15, wherein the first calibration image comprises a structured light pattern defining a plurality of elements. ### Claim 17 (depends on 14) The system of claim 14, wherein the processing element further determines the three-dimensional positions of the plurality of pattern elements by a depth detection algorithm. ### Claim 18 (depends on 14) The system of claim 14, wherein the processing element: detects the plurality of pattern elements using a blob detection algorithm; and further interpolates points that fall between the pattern elements within the first calibration image. ### Claim 19 (depends on 14) The system of claim 14, wherein the processing element creates a modified output image by transforming an input image by applying the distortion map to the input image, wherein the modified output image is corrected for distortion of the laser projector. ### Claim 20 A projection system comprising: a video projector projecting a first image onto a projection surface; and a laser projector projecting a second image onto the projection surface, wherein the laser projector is calibrated using a perspective projection matrix by modeling the laser projector as a pinhole device with a pinhole approximation to provide a first calibration approximation based on the first image and a non-linear mapping function to correct inaccuracies in the first calibration approximation resulting from the pinhole approximation and determine a distortion map to adjust for distortion intrinsic to the laser projector; wherein the second image overlays the first image on the projection surface. ### Claim 21 (depends on 20) The projection system of claim 20, wherein the second image is aligned with a first portion of the first image to provide an accent to the first image.