# US 2017/0287162 A1 · Method and system for scanning an object using an rgb-d sensor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0287162 A1
- Document ID: US20170287162A1
- Status: Published application
- Kind: A1
- Application number: 15/506,991
- Filed: 2014-08-29
- Published: 2017-10-05
- Earliest claimed: 2014-08-29
- Assignees: Toyota Motor Europe; Technische Universität München
- Applicants: Toyota Motor Europe; Technische Universitat Munchen
- Inventors: Zbigniew WASIK; Wadim KEHL; Nassir NAVAB
- Web page: https://patentgrep.com/patent/US20170287162A1

## CPC

- G06T 7/73: using feature-based methods
- G06T 7/579: from motion
- G06T 2200/04: involving 3D image data
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20072: Graph-based image processing
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method and system for scanning an object using an RGB-D sensor, the method includes: a plurality of elementary scans of the object using an RGB-D sensor and visual odometry, each elementary scan delivering a plurality of key frames associated with a pose of the sensor with respect to the object, and each elementary scan being associated with a position of the object; for each elementary scan, elaborating a three-dimensional model of the object using the plurality of key frames and poses of the scan; merging each three-dimensional model into a merged three-dimensional model of the object.

## Claims

### Claim at position 1

1-13. (canceled)

### Claim at position 2

14: A method for scanning an object comprising:
a plurality of elementary scans of the object using an RGB-D sensor and visual odometry, each elementary scan delivering a plurality of key frames associated with a pose of the sensor with respect to the object, and each elementary scan being associated with a position of the object, the position of the object being different between two different elementary scans;
for each elementary scan, elaborating a three-dimensional model of the object using the plurality of key frames and poses of the scan;
merging each three-dimensional model into a merged three-dimensional model of the object.

### Claim at position 3 (depends on 14)

15: The method according to claim 14, further comprising for each elementary scan a rotation of the object or a rotation of the sensor around the object, the object being in the position associated with the elementary scan.

### Claim at position 4 (depends on 15)

16: The method according to claim 15, wherein loop closure is detected after a revolution of the object or a revolution of the sensor around the object.

### Claim at position 5 (depends on 16)

17: The method according to claim 16, wherein a pose graph optimization is carried out on the poses of each elementary scan after detecting loop closure.

### Claim at position 6 (depends on 14)

18: The method according to claim 14, wherein the elaborating the three-dimensional model for each elementary scan is made using a variational technique using colored signed distance fields and an approximate L1 minimization.

### Claim at position 7 (depends on 14)

19: The method according to claim 14, wherein the elaborating the merged three-dimensional model of the object comprises a determination of the transformations that transforms each three-dimensional model into an initial three-dimensional model, the determination of the transformations comprising an alignment of each three-dimensional model into the initial three-dimensional model using an approximate L1 norm.

### Claim at position 8 (depends on 14)

20: The method according to claim 14, further comprising using the marching cube algorithm on the merged three-dimensional model of the object to obtain a three-dimensional mesh of the object.

### Claim at position 9 (depends on 14)

21: The method according to claim 14, wherein positions of the object during at least two elementary scans are different and display different portions of the object to the sensor.

### Claim at position 10 (depends on 14)

22: The method according to claim 14, further comprising a segmentation of each key frame to separate the object from the background on the key frame.

### Claim at position 11

23: A system for scanning an object, comprising:
an RGB-D sensor configured to be directed towards the object;
means for scanning the object in a position using the RGB-D sensor and visual odometry, delivering a plurality of key frames associated with a pose of the sensor with respect to the object;
means for elaborating a three-dimensional model of the object using a plurality of key frames and poses of a of a scan of the object;
means for merging a plurality of three-dimensional models into a merged three-dimensional model of the object.

### Claim at position 12 (depends on 23)

24: The system according to claim 23, further comprising means for rotating the object or rotating the sensor around the object.

### Claim at position 13 (depends on 24)

25: The system according to claim 24, further comprising means for detecting loop closure after a revolution of the object or a revolution of the sensor around the object.

### Claim at position 14 (depends on 25)

26: The system according to claim 25, further comprising means for carrying out a pose-graph optimization on the poses of each elementary scan after detecting loop closure.