# US 10,157,474 B2 · 3D recording device, method for producing a 3D image, and method for setting up a 3D recording device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,157,474 B2 - Document ID: US10157474B2 - Status: Granted - Kind: B2 - Application number: 14/896,154 - Filed: 2014-06-02 - Published: 2016-06-30 - Granted: 2018-12-18 - Earliest claimed: 2013-06-04 - Assignees: Testo AG - Applicants: Testo AG - Inventors: Jan-Friso Evers-Senne; Martin Stratmann; Patrick Zahn - Web page: https://patentgrep.com/patent/US10157474B2 ## CPC - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G01S 17/08: for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48) - G01S 17/88: Lidar systems specially adapted for specific applications - G06T 7/579: from motion - H04N 13/189: Recording image signals; Reproducing recorded image signals - H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00) - G01S 15/08: Systems for measuring distance only (indirect measurement G01S15/46) - G01S 15/88: Sonar systems specially adapted for specific applications (seismic or acoustic prospecting or detecting G01V1/00) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A 3D recording device (1) is provided including an image recording device (2), a distance measuring device (3), and an output unit (5). An image processing device (4) is used to calculate, for a sequence (8) of images (9, 10) recorded in different poses (15, 16, 17), groups (29, 30, 31, 32) of image elements (18, 19, 20, 21, 25, 26, 27, 28) corresponding to each other and to determine for each group (29, 30, 31, 32) a three-dimensional position indication (48, 49, 50) and to scale the three-dimensional position indication (48, 49, 50) with the aid of distance information (42) measured by the distance measuring device (3). ## Claims ### Claim 1 A 3D recording device (1) comprising: an image-recording device (2), a rangefinder (3) and an image-processing device (4), the image-processing device (4) is configured to: identify (33) at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) in a sequence (8) of at least two images (9, 10) recorded by the image-recording device (2), calculate (38) at least one three-dimensional position specification (50) in relation to the at least one group (29, 30, 31, 32) of the mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) from a respective image position (59, 60) of the image constituents (18, 19, 20, 21, 25, 26, 27, 28) from the group (29, 30, 31, 32) in the images (9, 10) of the sequence (8), calculate (38) a recording pose of the image-recording device (2) from the respective image position (59, 60) of each of the image constituents (18, 19, 20, 21, 25, 26, 27, 28) of the at least one group (29, 30, 31, 32), and then scale (46) a calculated at least one three-dimensional position specification (50) using measured distance information (42) that is measured by the rangefinder (3); and calculate further three-dimensional distance information associated with the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) for which no distance measurement is available, in at least one of an extrapolation or interpolation method (47) from the measured distance information (42) and the calculated at least one three-dimensional position specification (50). ### Claim 2 (depends on 1) The 3D recording device (1) as claimed in claim 1, further comprising an output unit (5) configured to output the at least one three-dimensional position specification which is scaled. ### Claim 3 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the image-processing device is configured to calculate further three-dimensional position specifications (48, 49) from the calculated three-dimensional position specifications (50) and recording poses (15, 16, 17). ### Claim 4 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the image-processing device (4) is configured to identify (33) the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) using a feature analysis (23), or the image-processing device (4) is configured to segment (55) the images (9, 10) into image segments, or the image-processing device (4) is configured to identify (33) the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) using a feature analysis (23) and to segment (55) the images (9, 10) into image segments. ### Claim 5 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein an actuator (57) is embodied to align a measurement direction of the rangefinder (3) in such a way that the measured distance information (42) is related to an image segment. ### Claim 6 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein at least one of a movement or acceleration sensor (58) is configured to detect a movement of the image-recording device (2). ### Claim 7 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the image-processing device (4) is configured to fit a geometric object (51) into the at least one scaled three-dimensional position specification. ### Claim 8 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the image-processing device (4) is configured to calculate at least one of a length, area or volume from the at least one three-dimensional position specification. ### Claim 9 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the rangefinder (3) is embodied as a laser rangefinder. ### Claim 10 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the image-processing device is configured to calculate further three-dimensional position specifications (48, 49) from the calculated three-dimensional position specifications (50) and recording poses (15, 16, 17) using a stereo method or from an interface calculated by a level set method. ### Claim 11 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the rangefinder (3) is configured to modify an alignment of at least one of a measurement direction or a measurement beam (52). ### Claim 12 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the rangefinder (3) is embodied to generate a measurement beam (52) in a spectral range detectable by the image-recording device (2). ### Claim 13 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the image-processing device (4) is configured to calculate a three-dimensional representation of the at least one group (29, 30, 31, 32) of the mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) at the at least one scaled three-dimensional position specification. ### Claim 14 (depends on 1) The 3D recording device (1) as claimed in claim 1, wherein the output unit (5) is configured to graphically output the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) at an image point corresponding to the three-dimensional position specification (48, 49, 50). ### Claim 15 A method for producing a 3D image, comprising recording a sequence (8) of at least two images (9, 10) of a scene (11) using an image-recording device (2) and identifying at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) in the recorded images (9, 10), calculating at least one three-dimensional position specification (50) relating to the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) in an image-processing device (4), calculating at least one recording pose (15, 16, 17) of the image-recording device (2) at a recording time from a respective image position (59, 60) of the image constituents (18, 19, 20, 21, 25, 26, 27, 28) from the at least one group (29, 30, 31, 32) in the images (9, 10) of the sequence (8), measuring at least one item of distance information (42) in relation to the scene (11) using a rangefinder (3), and then scaling (46) the three-dimensional position specification using the measured distance information (42), and calculating further three-dimensional distance information associated with the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) for which no distance measurement is available, in at least one of an extrapolation or interpolation method (47) from the measured distance information (42) and the calculated at least one three-dimensional position specification (50). ### Claim 16 (depends on 15) The method as claimed in claim 15, further comprising calculating further three-dimensional position specifications (48, 49) using a stereo method, or deriving the further three-dimensional position specifications (48, 49) from an interface calculated using a level set method. ### Claim 17 (depends on 15) The method as claimed in claim 15, further comprising identifying the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) using a feature analysis (23) or segmenting the recorded images (9, 10) using by the image-processing device (4). ### Claim 18 (depends on 15) The method as claimed in claim 15, further comprising measuring the distance information (50) in relation to a scene constituent (34, 35, 36, 37), which corresponds to the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28), automatically. ### Claim 19 (depends on 15) The method as claimed in claim 15, further comprising calculating the scaling (46) from the distance information (42) associated with the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28), or measuring distance information (42) in relation to each image segment of the images (9, 10). ### Claim 20 (depends on 15) The method as claimed in claim 15, further comprising measuring the distance information (42) using a measurement beam (52), which lies in a spectral range in which the image-recording device (2) records. ### Claim 21 (depends on 15) The method as claimed in claim 15, further comprising calculating the at least one three-dimensional position specification (48, 49, 50) using at least one of a SLAM method, or structure-from-motion method. ### Claim 22 (depends on 15) The method as claimed in claim 15, further comprising identifying a spatial region (59), in which further three-dimensional position specifications (48, 49) are to be calculated, and actuating the rangefinder (3) in such a way that the distance information (42) for the spatial region (59) is measured or a current recording pose (17) is calculated or estimated from previously calculated recording poses (15, 16) for the purposes of actuating the rangefinder (3). ### Claim 23 (depends on 15) The method as claimed in claim 15, further comprising fitting a geometric object (51) into the at least one three-dimensional position specification (48, 49, 50). ### Claim 24 (depends on 15) The method as claimed in claim 15, further comprising calculating at least one of a length, area or volume from the at least one three-dimensional position specification. ### Claim 25 (depends on 15) A method for setting up a 3D recording device for carrying out the method of claim 15, comprising connecting the rangefinder (3) to the image-recording device (2), measuring the at least one item of distance information (42) for each of the images (9, 10), determining an image position in the associated image (9, 10) for each said item of measured distance information (42), to which image position said distance information (42) relates, and calculating at least one parameter from the measured distance information (42) and the determined image positions, said parameter describing an alignment of the rangefinder (3) relative to the image-recording device (2). ### Claim 26 (depends on 25) The method as claimed in claim 25, further comprising projecting at least one of a known or an artificially generated scene with a measurement beam (52) of the rangefinder (3). ### Claim 27 (depends on 15) The method as claimed in claim 15, further comprising calculating further three-dimensional position specifications (48, 49) in relation to further image constituents before or after the scaling. ### Claim 28 (depends on 15) The method as claimed in claim 15, further comprising calculating a movement of the image-recording device (2) between two individual recordings using an output signal of at least one of a movement or acceleration sensor (58). ### Claim 29 (depends on 15) The method as claimed in claim 15, further comprising calculating a three-dimensional representation of the at least one group (29, 30, 31, 32) of mutually corresponding image constituents (18, 19, 20, 21, 25, 26, 27, 28) at the at least one scaled three-dimensional position specification. ### Claim 30 (depends on 15) The method as claimed in claim 15, further comprising outputting the at least one group (29, 30, 31, 32) at an image point corresponding to the at least one three-dimensional position specification (48, 49, 50) using an output unit (5).