# US 2021/0333384 A1 · Three-dimensional reconstruction device, three-dimensional reconstruction system, three-dimensional reconstruction method, and storage medium storing three-dimensional reconstruction program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0333384 A1 - Document ID: US20210333384A1 - Status: Published application - Kind: A1 - Application number: 17/371,374 - Filed: 2021-07-09 - Published: 2021-10-28 - Earliest claimed: 2019-01-16 - Assignees: Mitsubishi Electric Corporation - Applicants: Mitsubishi Electric Corporation - Inventors: Kento YAMAZAKI; Kohei OKAHARA; Jun MINAGAWA; Shinji MIZUNO; Shintaro SAKATA; Takumi SAKAKIBARA - Web page: https://patentgrep.com/patent/US20210333384A1 ## CPC - G01B 11/245: using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence) - G01S 13/426: {Scanning radar, e.g. 3D radar (G01S13/66 takes precedence)} - G01B 21/047: {Accessories, e.g. for positioning, for tool-setting, for measuring probes} - G01S 13/89: for mapping or imaging - G06T 7/55: from multiple images - G01B 2210/52: Combining or merging partially overlapping images to an overall image - G01S 17/89: for mapping or imaging - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A three-dimensional reconstruction device includes processing circuitry, to acquire first three-dimensional information representing a target object from a first sensor generating the first three-dimensional information by detecting the target object that is moving and to acquire second three-dimensional information representing an attention part of the target object from a second sensor generating the second three-dimensional information by detecting the attention part; to acquire first sensor information and second sensor information; to acquire first position posture information indicating a position and posture of the first sensor and to acquire second position posture information indicating a position and posture of the second sensor; and to reconstruct the three-dimensional information representing the attention part from the first three-dimensional information and the second three-dimensional information by using the first sensor information, the second sensor information, the first position posture information and the second position posture information. ## Claims ### Claim 1 A three-dimensional reconstruction device comprising: processing circuitry to acquire first three-dimensional information representing a target object from a first sensor arranged at a predetermined position and generating the first three-dimensional information by detecting the target object that is moving and to acquire second three-dimensional information representing an attention part of the target object from a second sensor provided to be movable and generating the second three-dimensional information by detecting the attention part; to acquire first sensor information indicating a property intrinsic to the first sensor and second sensor information indicating a property intrinsic to the second sensor; to acquire first position posture information indicating a position and posture of the first sensor and to acquire second position posture information indicating a position and posture of the second sensor; and to reconstruct the three-dimensional information representing the attention part from the first three-dimensional information and the second three-dimensional information by using the first sensor information, the second sensor information, the first position posture information and the second position posture information. ### Claim 2 (depends on 1) The three-dimensional reconstruction device according to claim 1, wherein the processing circuitry acquires the second three-dimensional information from the second sensor in real time. ### Claim 3 (depends on 1) The three-dimensional reconstruction device according to claim 1, wherein the processing circuitry acquires the second position posture information from the second sensor in real time. ### Claim 4 (depends on 1) The three-dimensional reconstruction device according to claim 1, further comprising a receiver that receives a radio signal, wherein the processing circuitry acquires the second three-dimensional information from the second sensor in real time via the receiver, and the processing circuitry acquires the second position posture information from the second sensor in real time via the receiver. ### Claim 5 (depends on 1) The three-dimensional reconstruction device according to claim 1, wherein the processing circuitry estimates the position and the posture of the second sensor based on movement information on the attention part indicated by the second three-dimensional information. ### Claim 6 (depends on 1) The three-dimensional reconstruction device according to claim 1, further comprising a storage that stores the three-dimensional information reconstructed by the processing circuitry. ### Claim 7 A three-dimensional reconstruction system comprising: a first sensor that is arranged at a predetermined position and generates first three-dimensional information representing a target object by detecting the target object that is moving; a second sensor that is provided to be movable and generates second three-dimensional information representing an attention part of the target object by detecting the attention part; and processing circuitry to acquire the first three-dimensional information and the second three-dimensional information; to acquire first sensor information indicating a property intrinsic to the first sensor and second sensor information indicating a property intrinsic to the second sensor; to acquire first position posture information indicating a position and posture of the first sensor and to acquire second position posture information indicating a position and posture of the second sensor; and to reconstruct the three-dimensional information representing the attention part from the first three-dimensional information and the second three-dimensional information by using the first sensor information, the second sensor information, the first position posture information and the second position posture information. ### Claim 8 (depends on 7) The three-dimensional reconstruction system according to claim 7, further comprising a movement apparatus that changes the position and the posture of the second sensor, wherein the movement apparatus controls the position and the posture of the second sensor based on the second three-dimensional information so that the attention part does not deviate from a detection range of the second sensor. ### Claim 9 (depends on 8) The three-dimensional reconstruction system according to claim 8, wherein the movement apparatus acquires the second three-dimensional information from the second sensor in real time. ### Claim 10 (depends on 8) The three-dimensional reconstruction system according to claim 8, wherein the movement apparatus acquires the second position posture information from the second sensor in real time. ### Claim 11 (depends on 8) The three-dimensional reconstruction system according to claim 8, wherein the movement apparatus controls the movement of the second sensor in consideration of the position of the first sensor. ### Claim 12 (depends on 8) The three-dimensional reconstruction system according to claim 8, wherein the movement apparatus executes control of permitting temporary deviation of the attention part from the detection range of the second sensor when the attention part is situated in a detection range of the first sensor. ### Claim 13 (depends on 7) The three-dimensional reconstruction system according to claim 7, wherein the processing circuitry estimates the position and the posture of the second sensor based on movement of the attention part in the second three-dimensional information. ### Claim 14 (depends on 7) The three-dimensional reconstruction system according to claim 7, wherein the processing circuitry acquires a plurality of pieces of the first three-dimensional information from a plurality of the first sensors, acquires a plurality of pieces of the first sensor information, acquires a plurality of pieces of the first position posture information, and reconstructs the three-dimensional information from the plurality of pieces of the first three-dimensional information and the second three-dimensional information by using the plurality of pieces of the first sensor information, the second sensor information, the plurality of pieces of the first position posture information and the second position posture information. ### Claim 15 (depends on 7) The three-dimensional reconstruction system according to claim 7, further comprising a storage that stores the three-dimensional information reconstructed by the processing circuitry. ### Claim 16 A three-dimensional reconstruction method comprising: acquiring first three-dimensional information representing a target object from a first sensor that is arranged at a predetermined position and generates the first three-dimensional information by detecting the target object that is moving; acquiring second three-dimensional information representing an attention part of the target object from a second sensor that is provided to be movable and generates the second three-dimensional information by detecting the attention part; acquiring first sensor information indicating a property intrinsic to the first sensor; acquiring second sensor information indicating a property intrinsic to the second sensor; acquiring first position posture information indicating a position and posture of the first sensor; acquiring second position posture information indicating a position and posture of the second sensor; and reconstructing the three-dimensional information representing the attention part from the first three-dimensional information and the second three-dimensional information by using the first sensor information, the second sensor information, the first position posture information and the second position posture information. ### Claim 17 A non-transitory computer-readable storage medium for storing a three-dimensional reconstruction program that causes a computer to execute processing comprising: acquiring first three-dimensional information representing a target object from a first sensor that is arranged at a predetermined position and generates the first three-dimensional information by detecting the target object that is moving; acquiring second three-dimensional information representing an attention part of the target object from a second sensor that is provided to be movable and generates the second three-dimensional information by detecting the attention part; acquiring first sensor information indicating a property intrinsic to the first sensor; acquiring second sensor information indicating a property intrinsic to the second sensor; acquiring first position posture information indicating a position and posture of the first sensor; acquiring second position posture information indicating a position and posture of the second sensor; and reconstructing the three-dimensional information representing the attention part from the first three-dimensional information and the second three-dimensional information by using the first sensor information, the second sensor information, the first position posture information and the second position posture information.