# US 12,112,431 B2 · Three dimensional mapping

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,431 B2
- Document ID: US12112431B2
- Status: Granted
- Kind: B2
- Application number: 17/827,053
- Filed: 2022-05-27
- Published: 2023-12-21
- Granted: 2024-10-08
- Earliest claimed: 2022-05-27
- Assignees: Quantarium Group, LLC
- Applicants: Quantarium Group, LLC
- Inventors: Andrei T. Aron; Bo Pintea; Yankang Chen; Clement Ifrim; Ioan Smintina
- Web page: https://patentgrep.com/patent/US12112431B2

## CPC

- G06T 17/05: Geographic models
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G01B 11/2513: {with several lines being projected in more than one direction, e.g. grids, patterns}
- G06T 7/248: {involving reference images or patches}
- G06T 7/579: from motion
- G06T 7/74: {involving reference images or patches}
- G06V 10/235: {based on user input or interaction}
- H04N 23/60: Control of cameras or camera modules
- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/632: {for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters}
- H04N 23/633: {for displaying additional information relating to control or operation of the camera}
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20101: Interactive definition of point of interest, landmark or seed
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods are disclosed for three dimensional mapping. In certain embodiments, a method may comprise executing a three dimensional mapping operation via a computing system, including capturing an image sequence of a plurality of physical surfaces via an image capture device, and receiving a user input corresponding to the image sequence, the user input identifying a location of a surface edge from the plurality of physical surfaces. The three dimensional mapping operation may further include identifying a plurality of feature points within the image sequence based on a projected light pattern on the plurality of physical surfaces, and generating a three dimensional map of the plurality of physical surfaces based on the image sequence, the user input, and the plurality of feature points.

## Claims

### Claim 1

A method comprising:
executing a three dimensional mapping operation via a computing system, including:
capturing an image sequence of a plurality of physical surfaces via an image capture device;
receiving a user input corresponding to the image sequence, the user input identifying a location of a surface edge from the plurality of physical surfaces by placing an indicator of the location of the surface edge in the image sequence;
tracking a plurality of feature points between individual images in the image sequence based on a projected light pattern situated to create a consistent artificial texture pattern on the plurality of physical surfaces; and
generating a three dimensional map of the plurality of physical surfaces based on the image sequence, the user input identifying the location of the surface edge, and the plurality of feature points.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
generating the three dimensional map includes:
identifying a feature point in a first image of the image sequence;
identifying the feature point in a second image of the image sequence; and
determining motion of the image capture device based on a difference in position between the feature point in the first image and the feature point in the second image.

### Claim 3 (depends on 2)

The method of claim 2 further comprising:
determining the motion of the image capture device via application of a simultaneous localization and mapping (SLAM) algorithm.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
generating the three dimensional map via application of a structure from motion (SfM) algorithm.

### Claim 5 (depends on 1)

The method of claim 1 further comprising:
obtaining location information based on a position sensor for the image capture device; and
generating the three dimensional map further based on the location information.

### Claim 6 (depends on 5)

The method of claim 5 further comprising the location information includes global positioning system (GPS) coordinates.

### Claim 7 (depends on 5)

The method of claim 5 further comprising the position sensor includes an accelerometer.

### Claim 8 (depends on 5)

The method of claim 5 further comprising the position sensor includes a gyroscope.

### Claim 9 (depends on 1)

The method of claim 1 further comprising:
the computing system includes a smartphone incorporating the image capture device;
displaying the image sequence on a screen of the smartphone; and
the user input includes tap inputs on the screen corresponding to a location of the surface edge in a currently displayed image of the image sequence.

### Claim 10

A memory device storing instructions that, when executed, cause a processor to perform a method comprising:
generating a three dimensional map of a physical space, including:
capturing an image sequence of a plurality of physical surfaces via an image capture device;
tracking a plurality of feature points between individual images in the image sequence based on a projected light pattern situated to create a consistent artificial texture pattern on the plurality of physical surfaces; and
generating the three dimensional map of the plurality of physical surfaces based on the image sequence and the plurality of feature points.

### Claim 11 (depends on 10)

The memory device of claim 10 storing the instructions that, when executed, cause the processor to perform the method further comprising:
receiving a user input corresponding to the image sequence, the user input identifying a location of a surface edge from the plurality of physical surfaces by placing an indicator of the location of the surface edge in the image sequence; and
generating the three dimension map further based on the user input.

### Claim 12 (depends on 10)

The memory device of claim 10 storing the instructions that, when executed, cause the processor to perform the method further comprising:
generating the three dimensional map further includes:
identifying a feature point, from the plurality of feature points, in a first image of the image sequence;
identifying the feature point in a second image of the image sequence, the second image captured from a different physical position than the first image; and
generating a recreation of the physical space based on a difference in position between the feature point in the first image and the feature point in the second image.

### Claim 13 (depends on 12)

The memory device of claim 12 storing the instructions that, when executed, cause the processor to perform the method further comprising:
generating the three dimensional map via application of a structure from motion (SfM) algorithm.

### Claim 14 (depends on 12)

The memory device of claim 12 storing the instructions that, when executed, cause the processor to perform the method further comprising:
presenting a user interface on a touchscreen device;
operating the image camera device in response to a user instruction received via the user interface; and
capturing the image sequence as a set of streaming images automatically captured in sequence.

### Claim 15 (depends on 10)

The memory device of claim 10 storing the instructions that, when executed, cause the processor to perform the method further comprising:
obtaining location information based on a position sensor for the image capture device; and
generating the three dimensional map further based on the location information.

### Claim 16

An apparatus comprising:
a processor configured to execute a three dimensional mapping operation, including:
capture an image sequence of a plurality of physical surfaces via an image capture device;
receive a user input corresponding to the image sequence, the user input identifying a location of a surface edge from the plurality of physical surfaces by placing an indicator of the location of the surface edge in the image sequence;
track a plurality of feature points between individual images in the image sequence based on a projected light pattern situated to create a consistent artificial texture pattern on the plurality of physical surfaces; and
generate a three dimensional map of the plurality of physical surfaces based on the image sequence, the user input, and the plurality of feature points.

### Claim 17 (depends on 16)

The apparatus of claim 16 further comprising:
a mobile computing device including:
the processor;
the image capture device; and
a position sensor configured to generate location information for the mobile computing device; and
the processor is further configured to generate the three dimensional map further based on the location information.

### Claim 18 (depends on 17)

The apparatus of claim 17 further comprising:
the position sensor includes at least one of a global positioning system (GPS) sensor, an accelerometer, and a gyroscope.

### Claim 19 (depends on 16)

The apparatus of claim 16 comprising the processor further configured to:
generate the three dimensional map, including:
identify a feature point in a first image of the image sequence based on the user input;
identify the feature point in a second image of the image sequence;
determine motion of the image capture device based on a difference in position between the feature point in the first image and the feature point in the second image; and
determine a position of the plurality of physical surfaces based on the motion of the image capture device and the sequence of images.

### Claim 20 (depends on 16)

The apparatus of claim 16 comprising the processor further configured to:
determine the motion of the image capture device via application of a simultaneous localization and mapping (SLAM) algorithm; and
generate the three dimensional map via application of a structure from motion (SfM) algorithm.