# US 10,371,518 B2 · Surveying system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,371,518 B2 - Document ID: US10371518B2 - Status: Granted - Kind: B2 - Application number: 16/197,071 - Filed: 2018-11-20 - Published: 2019-03-28 - Granted: 2019-08-06 - Earliest claimed: 2016-12-05 - Assignees: Hexagon Technology Center GmbH - Applicants: Hexagon Technology Center GmbH - Inventors: Knut Siercks; Bernhard Metzler; Elmar Vincent Van Der Zwan; Thomas Fidler; Roman Parys; Alexander Velizhev; Jochen Scheja - Web page: https://patentgrep.com/patent/US10371518B2 ## CPC - G01C 15/006: {Detectors therefor} - G01C 11/02: Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures - G01C 11/04: Interpretation of pictures - G01C 15/002: {Active optical surveying means (optical plumbing G01C15/105)} - G01C 25/005: {initial alignment, calibration or starting-up of inertial devices} - G01S 19/13: Receivers - G01S 19/14: specially adapted for specific applications - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06T 7/0004: {Industrial image inspection} - G06T 7/579: from motion - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - H04N 23/51: Housings - H04N 23/55: Optical parts specially adapted for electronic image sensors; Mounting thereof - H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices - H04N 23/63: by using electronic viewfinders - H04N 23/635: {Region indicators; Field of view indicators} - H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30244: Camera pose - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A system is disclosed that comprises a camera module and a control and evaluation unit. The camera module is designed to be attached to the surveying pole and comprises at least one camera for capturing images. The control and evaluation unit has stored a program with program code so as to control and execute a functionality in which a series of images of the surrounding is captured with the at least one camera; a SLAM-evaluation with a defined algorithm using the series of images is performed, wherein a reference point field is built up and poses for the captured images are determined; and, based on the determined poses, a point cloud comprising 3D-positions of points of the surrounding can be computed by forward intersection using the series of images, particularly by using dense matching algorithm. ## Claims ### Claim 1 A surveying subsystem comprising a camera module and a control and evaluation unit, wherein the surveying subsystem is adapted to be used as part of a surveying system that is adapted to determine positions of a position measuring resource, particularly wherein the surveying system further comprising a hand-carried surveying pole and the position measuring resource being mounted on the hand-carried surveying pole: the camera module comprising at least one camera for capturing images, particularly wherein the camera module is designed to be attached to the hand-carried surveying pole, the control and evaluation unit having stored a program with program code so as to control and execute a spatial representation generation functionality in which—when moving along a path through a surrounding— a series of images of the surrounding is captured with the at least one camera, the series comprising an amount of images captured with different poses of the camera, the poses representing respective positions and orientations of the camera, a SLAM-evaluation with a defined algorithm using the series of images is performed, wherein a plurality of respectively corresponding image points are identified in each of several sub-groups of images of the series of images and, based on resection and forward intersection using the plurality of respectively corresponding image points, a reference point field is built up comprising a plurality of reference points of the surrounding, wherein coordinates of the reference points are derived, and the poses for the images are determined, based on the determined poses, a point cloud comprising 3D-positions of points of the surrounding is computed by forward intersection using the images of the series of images, determined positions of the position measuring resource for points that have been adopted on the path are received by the control and evaluation unit from the surveying system and the point cloud is scaled, and particularly geo-referenced, with help of the received determined positions. ### Claim 2 (depends on 1) A subsystem according to claim 1, wherein, the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that the point cloud is generated: spatially inclusive and comprehensive across the whole surrounding and/or with comparatively low resolution of the 3d-information across the surrounding, thus providing comparatively fast processing of the point cloud. ### Claim 3 (depends on 1) The subsystem according to claim 1, wherein: the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that a graphical reproduction is generated for the scaled point cloud, the graphical reproduction being displayable by display means of the surveying system, thus providing for a direct feedback to a user about already acquired data, so that the already acquired data can be checked regarding its completeness. ### Claim 4 (depends on 1) The subsystem according to claim 1, wherein, the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that position information is derived for a single point selected in at least one image of the series of images, wherein a subset of images with determined poses related to the selected point is automatically identified from the series of images, particularly all images in which the selected point appears, and the position information is calculated based on the subset, particularly after the point was manually selected by a user. ### Claim 5 (depends on 1) The subsystem according to claim 1, wherein, the control and evaluation unit is configured so that the spatial representation generation functionality is controlled and executed in such a way, that the point cloud is processed covering: the surrounding as a whole as commonly appearing at least in pairs of images of series of images, thus providing for a global representation with comparatively low point-to-point resolution, and/or a defined region of the surrounding as commonly appearing at least in pairs of images of series of images, thus providing for a regional representation with higher point-to-point resolution compared to the global representation, particularly wherein the point-to-point resolution of the point cloud is automatically adapted depending on the size of the region so that processing time fulfils a defined threshold. ### Claim 6 (depends on 1) The subsystem according to claim 1, wherein, the position measuring resource comprises a GNSS-antenna or a retro-reflector. ### Claim 7 (depends on 1) The subsystem according to claim 1, wherein, orientations of the at least one camera, particularly in all three rotational degrees of freedom, in particular position and orientation in six degrees of freedom, are derived based on one or more of the determined poses and further based on: data from an inertial measuring unit of the camera module, and/or a multitude of determined positions of the position measuring resource, particular a travelling history for the moved path. ### Claim 8 (depends on 1) A surveying system comprising: a hand-carried surveying pole, a position measuring resource being mounted on the surveying pole, wherein positions of the position measuring resource are determinable by the surveying system, and a surveying subsystem according to claim 1. ### Claim 9 A surveying subsystem comprising a camera module and a control and evaluation unit to be used as part of a surveying system that is adapted to determine positions with use of a surveying pole: the camera module being designed to be attached to the surveying pole in a known distance to a bottom end of the pole and comprising at least one camera for capturing images, the control and evaluation unit having stored a program with program code so as to control and execute a scaling functionality in which—when moving along a path through a surrounding— a series of images of the surrounding is captured with the at least one camera, a surrounding ground appearing in the images, the series comprising an amount of images captured with different poses of the camera, the poses representing respective positions and orientations of the camera; a SLAM-evaluation with a defined algorithm using the series of images is performed, wherein a plurality of respectively corresponding image points are identified in each of several sub-groups of images of the series of images and, based on resection and forward intersection using the plurality of respectively corresponding image points, a reference point field is built up comprising a plurality of reference points of the surrounding, wherein coordinates of the reference points are derived, and the poses for the images are determined; based on the determined poses, a point cloud comprising 3D-positions of points of the surrounding is computed by forward intersection using the series of images, particularly by using dense matching algorithm; a distance from the camera to the ground is determined based on the known distance from the camera to the bottom end; and the point cloud is scaled based on the determined distance. ### Claim 10 (depends on 9) The subsystem according to claim 9, wherein: scaling the point cloud comprises determining 3D-positions of points on the ground based on the determined distance. ### Claim 11 (depends on 9) The subsystem according to claim 9, wherein, determining the distance from the camera to the ground comprises deriving an orientation of the camera. ### Claim 12 (depends on 9) The subsystem according to claim 9, wherein, the orientation, particularly in all three rotational degrees of freedom, in particular position and orientation in six degrees of freedom, is derived from the poses and optionally further based on: data from an inertial measuring unit of the camera module, and/or a multitude of determined positions of a position measuring resource being mounted on the hand-carried surveying pole, the determined positions being determined for points that have been adopted on the path and being received by the control and evaluation unit from the surveying system. ### Claim 13 (depends on 9) The subsystem according to claim 9, wherein, the camera module comprises at least four cameras arranged relative to each other so that panoramic images with a field of view of 360° in azimuthal direction are capturable. ### Claim 14 (depends on 13) The subsystem according to claim 13, wherein, the at least four cameras are fixedly arranged in a common housing of the camera module, in particular wherein also an inertial measuring unit is fixedly integrated into the common housing of the camera module. ### Claim 15 (depends on 9) A surveying system comprising: a hand-carried surveying pole, a position measuring resource being mounted on the surveying pole, wherein positions of the position measuring resource are determinable by the surveying system, and a surveying subsystem according to claim 9.