# US 2022/0046221 A1 · Generating textured three-dimensional meshes using two-dimensional scanner and panoramic camera

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0046221 A1
- Document ID: US20220046221A1
- Status: Published application
- Kind: A1
- Application number: 17/325,956
- Filed: 2021-05-20
- Published: 2022-02-10
- Earliest claimed: 2020-08-07
- Assignees: Faro Technologies, Inc.
- Applicants: FARO Technologies, Inc.
- Inventors: Mark Brenner; Aleksej Frank; Ahmad Ramadneh; Mufassar Waheed; Oliver Zweigle
- Web page: https://patentgrep.com/patent/US20220046221A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- H04N 13/261: with monoscopic-to-stereoscopic image conversion
- G01P 15/097: by vibratory elements
- G06T 15/04: Texture mapping
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 13/275: from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G})

## Abstract

Techniques are described for converting a 2D map into a 3D mesh. The 2D map of the environment is generated using data captured by a 2D scanner. Further, a set of features is identified from a subset of panoramic images of the environment that are captured by a camera. Further, the panoramic images from the subset are aligned with the 2D map using the features that are extracted. Further, 3D coordinates of the features are determined using 2D coordinates from the 2D map and a third coordinate based on a pose of the camera. The 3D mesh is generated using the 3D coordinates of the features.

## Claims

### Claim 1

A system comprising:
a scanner configured to capture a 2D map of an environment;
a camera configured to capture a plurality of panoramic images of the environment from respective locations in the environment;
an inertial measurement unit (IMU) configured to determine a pose of the scanner, and a pose of the camera; and
one or more processors operably coupled to the scanner, the IMU, and the camera, the one or more processors being responsive to executable instructions for converting the 2D map into a 3D mesh, the converting comprising:
generating the 2D map of the environment using data captured by the scanner;
identifying a plurality of features from a subset of the panoramic images captured by the camera;
aligning the subset of the panoramic images with the 2D map using the features that are extracted;
determining 3D coordinates of the features using 2D coordinates from the 2D map and a third coordinate based on a pose of the camera; and
generating a 3D mesh using the 3D coordinates of the features.

### Claim 2 (depends on 1)

The system of claim 1, wherein the converting further comprises applying the subset of the panoramic images as textures on the 3D mesh.

### Claim 3 (depends on 1)

The system of claim 1, wherein the converting further comprises scaling the 3D mesh using the 2D map.

### Claim 4 (depends on 1)

The system of claim 1, wherein the camera is coupled with the scanner at a predetermined offset.

### Claim 5 (depends on 1)

The system of claim 1, wherein the camera and the scanner are coupled with a platform at predetermined positions on the platform.

### Claim 6 (depends on 1)

The system of claim 1, wherein the features comprise at least one from a corner, an edge, and a predetermined shape.

### Claim 7 (depends on 1)

The system of claim 1, wherein the scanner is a 2D scanner disposed in a body of a housing, the housing being sized to be carried by a single person during operation, the body having a first plane extending there through.

### Claim 8

A method for generating a three-dimensional (3D) mesh of an environment using a two-dimensional (2D) scanner and a camera, the method comprising:
generating a 2D map of the environment using data captured using the 2D scanner;
capturing, using the camera, a plurality of panoramic images of the environment from respective locations in the environment;
identifying a plurality of features from a subset of the panoramic images captured by the camera;
aligning the subset of the panoramic images with the 2D map using the features that are extracted;
determining 3D coordinates of the features using 2D coordinates from the 2D map and a third coordinate based on a pose of the camera; and
generating the 3D mesh using the 3D coordinates of the features.

### Claim 9 (depends on 8)

The method of claim 8, further comprising applying the subset of the panoramic images as textures on the 3D mesh.

### Claim 10 (depends on 8)

The method of claim 8, wherein the panoramic images are captured at locations different from locations at which the 2D scanner captures data for the 2D map.

### Claim 11 (depends on 8)

The method of claim 8, wherein the camera is coupled with the scanner at a predetermined offset.

### Claim 12 (depends on 8)

The method of claim 8, wherein the camera and the scanner are coupled with a platform at predetermined positions on the platform.

### Claim 13 (depends on 8)

The method of claim 8, wherein the features comprise at least one from a corner, an edge, and a predetermined shape.

### Claim 14 (depends on 8)

The method of claim 8, wherein the 2D scanner is disposed in a body of a housing, the housing being sized to be carried by a single person during operation, the body having a first plane extending there through.

### Claim 15

A computer program product comprising a memory device with computer executable instructions stored thereon, which when executed by one or more processing units causes the one or more processing units to execute a method for generating a three-dimensional (3D) mesh of an environment, the method comprising:
generating a 2D map of the environment using data captured using a 2D scanner;
capturing, using a camera, a plurality of panoramic images of the environment from respective locations in the environment;
identifying a plurality of features from a subset of the panoramic images captured by the camera;
aligning the subset of the panoramic images with the 2D map using the features that are extracted;
determining 3D coordinates of the features using 2D coordinates from the 2D map and a third coordinate based on a pose of the camera; and
generating the 3D mesh using the 3D coordinates of the features.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the method comprises applying the subset of the panoramic images as textures on the 3D mesh.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein the camera is coupled with the scanner at a predetermined offset.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein the camera and the scanner are coupled with a platform at predetermined positions on the platform.

### Claim 19 (depends on 15)

The computer program product of claim 15, wherein the features comprise at least one from a corner, an edge, and a predetermined shape.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein the 2D scanner is disposed in a body of a housing, the housing being sized to be carried by a single person during operation, the body having a first plane extending there through.
