# US 2022/0284608 A1 · Optical data processing apparatus, optical data processing method and optical data processing program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0284608 A1 - Document ID: US20220284608A1 - Status: Published application - Kind: A1 - Application number: 17/653,632 - Filed: 2022-03-04 - Published: 2022-09-08 - Earliest claimed: 2021-03-08 - Assignees: Topcon Corporation - Applicants: Topcon Corporation - Inventors: You Sasaki - Web page: https://patentgrep.com/patent/US20220284608A1 ## CPC - G06T 7/33: using feature-based methods - G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light - G01C 21/1652: {with ranging devices, e.g. LIDAR or RADAR} - G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G01S 17/89: for mapping or imaging - G06T 3/14: Transformations for image registration, e.g. adjusting or mapping for alignment of images - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/16: {using multiple overlapping images; Image stitching} - G06V 10/46: Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56) - G06T 2207/10016: Video; Image sequence - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Synchronizing between multiple optical data can be maintained by a facilitated method. An optical data processing operational unit 108 includes an optical data obtaining part 301 which obtains laser scan data by a laser scanner mounted on a moving body and image data of a photographed image photographed by a camera mounted on the moving body; and a delay time obtaining part 305 which calculates Δt when the point cloud image viewed from the viewpoint at time a specific time T made based on the laser scan data is obtained and the camera photographs with a delay Δt when the camera is ordered to photograph at time T; wherein relationships of exterior orientation elements of the laser scanner and the camera are known, and Δt is calculated in a condition in which difference in overlapping degree of the point cloud image and the photographed image is minimal. ## Claims ### Claim 1 An optical data processing apparatus comprising: an optical data obtaining part which obtains a laser scan point cloud obtained by a laser scanner mounted on a moving body in a moving condition and image data of a photographed image photographed by a camera mounted on the moving body in a moving condition, a delay time obtaining part that obtains Δt when the camera photographs with a delay Δt when the camera is commanded to take a photograph at a time T, and a projection part which makes a projection image, the projection image made by overlapping and projecting a point cloud image of which the laser scan point cloud is viewed from a specific viewpoint and the photographed image in which directions of viewing lines are aligned, wherein relationships of exterior orientation elements of the laser scanner and the camera in the moving body are known, the projections are performed multiple times by changing a viewpoint position of at least one of the point cloud image and the photographed image so that multiple projection images are made, and Δt is calculated in conditions in which difference in overlapping degree of the point cloud image and the photographed image in the multiple made projection images is minimal or is not greater than a threshold value. ### Claim 2 (depends on 1) The optical data processing apparatus according to claim 1, wherein the apparatus further comprises a camera position calculating part which calculates camera position in photographing of the photographed image by a single photograph orientation based on correspondence relationships of the laser scan point cloud and the photographed image, a reference value of Δt is calculated based on difference between a time at which a command for photographing the photographed image is output and a time the camera position is calculated, and a range of viewpoint positions is selected based on the reference value of Δt. ### Claim 3 (depends on 2) The optical data processing apparatus according to claim 2, wherein a specific range having a center position corresponding to the reference value of Δt is selected as the range of viewpoints. ### Claim 4 (depends on 1) The optical data processing apparatus according to claim 1, wherein Δt is obtained at regular intervals repeatedly. ### Claim 5 (depends on 1) The optical data processing apparatus according to claim 1, wherein each time a setting of a camera is changed, Δt is obtained. ### Claim 6 An optical data processing method comprising: a step of obtaining a laser scan point cloud obtained by a laser scanner mounted on a moving body in a moving condition, and image data of a photographed image photographed by a camera mounted on the moving body in a moving condition; a step of obtaining Δt when the camera photographs with a delay Δt when the camera is commanded to take a photograph at time T; and a step of making a projection image, the projection image made by overlapping and projecting a point cloud image in which the laser scan point cloud is viewed from a specific viewpoint and the photographed image in which directions of viewing lines are aligned, wherein relationships of exterior orientation elements of the laser scanner and the camera on the moving body are known, projection is performed multiple times by changing viewpoint position of at least one of the point cloud image and the photographed image so that multiple projection images are made, and Δt is calculated in a condition in which difference in overlapping degree of the point cloud image and the photographed image in the multiple made projection images is minimal or is not greater than a threshold value. ### Claim 7 A non-transitory computer recording medium storing computer executable instructions that, when executed by a computer processor, cause the computer processor to perform operations regarding an optical data processing program comprising: obtaining a laser scan point cloud obtained by a laser scanner mounted on a moving body in a moving condition and image data of a photographed image photographed by a camera mounted on the moving body in a moving condition, obtaining Δt in a case in which the camera photographs with a delay Δt when the camera is commanded to take a photograph at time T, and making a projection image, the projection image made by overlapping and projecting a point cloud image in which the laser scan point cloud is viewed from a specific viewpoint and the photographed image in which directions of viewing lines are aligned, wherein relationships of exterior orientation elements of the laser scanner and the camera on the moving body are known, projection is performed multiple times by changing viewpoint position of at least one of the point cloud image and the photographed image so that multiple projection images are made, and Δt is calculated in a condition in which difference in overlapping degree of the point cloud image and the photographed image in the multiple made projection images is minimal or is not greater than a threshold value.