# US 2019/0156568 A1 · System and method of scanning an environment and generating two dimensional images of the environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0156568 A1
- Document ID: US20190156568A1
- Status: Published application
- Kind: A1
- Application number: 16/179,357
- Filed: 2018-11-02
- Published: 2019-05-23
- Earliest claimed: 2017-11-21
- Assignees: Faro Technologies, Inc.
- Applicants: FARO Technologies, Inc.
- Inventors: Aleksej Frank; Matthias Wolke; Oliver Zweigle
- Web page: https://patentgrep.com/patent/US20190156568A1

## CPC

- G06T 17/05: Geographic models
- G06T 1/0007: {Image acquisition}
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system and method for scanning an environment and generating an annotated 2D map is provided. The system includes a 2D scanner having a light source, an image sensor and a first controller. The first controller determines a distance value to at least one of the object points. The system further includes a 360° camera having a movable platform, and a second controller that merges the images acquired by the cameras to generate an image having a 360° view in a horizontal plane. The system also includes processors coupled to the 2D scanner and the 360° camera. The processors are responsive to generate a 2D map of the environment based at least in part on a signal from an operator and the distance value. The processors being further responsive for acquiring a 360° image and integrating it at a location on the 2D map.

## Claims

### Claim 1

A system for scanning an environment and generating an annotated two-dimensional (2D) map, the system comprising:
a 2D scanner having a light source, an image sensor and a first controller, the light source steers a beam of light within the first plane to illuminate object points in the environment, the image sensor is arranged to receive light reflected from the object points, the first controller being operable to determine a distance value to at least one of the object points, the 2D scanner further having an inertial measurement unit, the inertial measurement unit generating a signal in response a change in position or orientation of the 2D scanner, the 2D scanner being sized and weighted to be carried and operated by a single person;
a 360° camera having a movable platform, the 360° camera having a plurality of cameras and a second controller, the second controller being operable to merge the images acquired by the plurality of cameras to generate an image having a 360° view in a horizontal plane; and
one or more processors operably coupled to the 2D scanner and the 360° camera, the one or more processors being responsive to non-transient executable instructions for generating a 2D map of the environment in response to an activation signal from an operator and based at least in part on the distance value and the signal, the one or more processors being further responsive for acquiring the 360° image and integrating the 360° image at a location on the 2D map that corresponds to the location where the image was acquired.

### Claim 2 (depends on 1)

The system of claim 1, wherein the one or more processors are further responsive to generating a point cloud based at least in part on the acquisition of at least two 360° images.

### Claim 3 (depends on 1)

The system of claim 1, wherein the one or more processors are further responsive to extracting planes from the 360° image and matching planes in the image with lines on the 2D map.

### Claim 4 (depends on 3)

The system of claim 3, wherein the one or more processors are further responsive to registering a first 360° image with a second 360° image.

### Claim 5 (depends on 4)

The system of claim 4, wherein the one or more processors are further responsive to generating a point cloud based at least in part on the registration of the first 360° image with the second 360° image and the location of the first 360° image and second 360° image on the 2D map.

### Claim 6 (depends on 5)

The system of claim 5, wherein the generation of the point cloud is further based at least in part on dense image matching.

### Claim 7 (depends on 1)

The system of claim 1, wherein the 360° image is a spherical projection.

### Claim 8 (depends on 1)

The system of claim 1, wherein the 360° image is a cylindrical projection.

### Claim 9

A method for generating a two-dimensional (2D) image of an environment, the method comprising:
scanning the environment with a 2D scanner having a light source, an image sensor, and an inertial measurement unit, the light source steers a beam of light within the first plane to illuminate object points in the environment, the image sensor is arranged to receive light reflected from the object points, the 2D scanner being sized and weighted to be carried and operated by a single person;
determine a distance value to at least one of the object point with the 2D scanner;
determining a change in position or orientation of the 2D scanner with the inertial measurement unit;
acquiring a plurality of images with a 360° camera, the 360° camera being coupled to movable platform and having a plurality of cameras, each of the plurality of images being acquired by one of the plurality of cameras;
merging the plurality of images to generate an image having a 360° view in a horizontal plane;
generating a 2D map of the environment in response to an activation signal from an operator and based at least in part on the distance value and the signal; and
integrating the 360° image at a location on the 2D map that corresponds to the location where the plurality of images were acquired.

### Claim 10 (depends on 9)

The method of claim 9, further comprising generating a point cloud based at least in part on the acquisition of at least two 360° images.

### Claim 11 (depends on 9)

The method of claim 9, further comprising extracting planes from the panoramic image and matching planes in the image with lines on the 2D map.

### Claim 12 (depends on 11)

The method of claim 11, further comprising registering a first 360° image with a second 360° image.

### Claim 13 (depends on 12)

The method of claim 12, further comprising generating a point cloud based at least in part on the registration of the first 360° image with the second 360° image and the location of the first 360° image and second 360° image on the 2D map.

### Claim 14 (depends on 13)

The method of claim 13, wherein the generation of the point cloud is further based at least in part on dense image matching.

### Claim 15 (depends on 9)

The method of claim 9, wherein the 360° image is a spherical projection.

### Claim 16 (depends on 9)

The method of claim 9, wherein the 360° image is a cylindrical projection.

### Claim 17 (depends on 9)

The method of claim 9, wherein the generating the 2D map includes generating a grid map that includes a plurality of cells, each of the plurality of cells indicating a probability of a cell area being occupied.

### Claim 18

A system of generating a two-dimensional (2D) image of an environment, the system comprising:
one or more processors;
a 2D scanner sized and weighted to be carried by a single person, having a first light source, an image sensor, and an inertial measurement unit, the first light source steers a beam of light within a first plane to illuminate object points in the environment, the image sensor is arranged to receive light reflected from the object points;
a 3D measurement device having a 360° camera coupled to movable platform, the 360° camera having a plurality of cameras and a second controller, the second controller being operable to merge the images acquired by the plurality of cameras to generate an image having a 360° view in a horizontal plane, the 3D scanner being movable from a first position to a second position on the movable platform;
wherein the one or more processors are responsive to non-transient executable instructions which when executed by the one or more processors to:
cause the 2D scanner to acquire a plurality of two-dimensional coordinates of points on surfaces in the environment;
cause the 3D scanner at the first position to acquire a first 360° image of the environment;
generate a 2D map based at least in part on the plurality of two-dimensional coordinates of points; and
integrating the first 360° image onto the 2D map.

### Claim 19 (depends on 18)

The system of claim 18, wherein the executable instructions further comprise causing the 3D scanner at the second position to acquire a second 360° image of the environment.

### Claim 20 (depends on 19)

The system of claim 19, wherein the executable instructions further comprise registering the first 360° image to the second 360° image.

### Claim 21 (depends on 20)

The system of claim 20, further comprising generating a point cloud based at least in part on the first 360° image and the second 360° image.

### Claim 22 (depends on 21)

The system of claim 21, wherein the point cloud is generated using dense image matching.

### Claim 23 (depends on 21)

The system of claim 21, further comprising creating an electronic link between the point cloud and the 2D map.
