# US 10,482,616 B2 · 3D model reconstruction method, electronic device, and non-transitory computer readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,616 B2 - Document ID: US10482616B2 - Status: Granted - Kind: B2 - Application number: 15/943,663 - Filed: 2018-04-02 - Published: 2018-10-18 - Granted: 2019-11-19 - Earliest claimed: 2017-04-17 - Assignees: HTC Corporation - Applicants: HTC Corporation - Inventors: Jui-Hsuan Chang; Yu-Cheng Hsu; Cheng-Yuan Shih; Sheng-Yen Lo; Hung-Yi Yang - Web page: https://patentgrep.com/patent/US10482616B2 ## CPC - G06T 7/55: from multiple images - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A 3D model reconstruction method includes: providing, by one or more laser emitters, laser beams; capturing, by a depth camera on an electronic device, a depth data of a target object when that the electronic device moves around the target object; detecting, by one or more light sensors on the electronic device, the laser beams emitted by the one or more laser emitters to obtain a camera pose initial value of the depth camera accordingly; and performing, by a processing circuit, a 3D reconstruction of the target object using the depth data based on the camera pose initial value to output a 3D model of the target object. ## Claims ### Claim 1 A 3D model reconstruction method, comprising: providing, by one or more laser emitters, a plurality of laser beams; capturing, by a depth camera on an electronic device, a depth data of a target object when that the electronic device moves around the target object; detecting, by one or more light sensors on the electronic device, the laser beams to obtain a camera pose initial value of the depth camera accordingly; and performing, by a processing circuit, a 3D reconstruction of the target object using the depth data based on the camera pose initial value to output a 3D model of the target object, wherein to obtain the camera pose initial value of the depth camera comprises: obtaining, by the processing circuit, a tracker pose of the electronic device according to the laser beams detected by the one or more light sensors; and obtaining, by the processing circuit, the camera pose initial value according to the tracker pose, wherein the tracker pose is transferred from tracker coordinate to camera coordinate to obtain the camera pose initial value of the depth camera. ### Claim 2 (depends on 1) The 3D model reconstruction method of claim 1, further comprising: estimating, by the processing circuit, a camera pose between a first time point and a second time point based on the depth data at a first time point, the depth data at a second time point, and the camera pose initial value. ### Claim 3 (depends on 2) The 3D model reconstruction method of claim 2, further comprising: converting, by the processing circuit, the depth data obtained at the first time point to a first cloud of points by applying a depth map conversion; converting, by the processing circuit, the depth data obtained at the second time point to a second cloud of points by applying the depth map conversion; and performing, by the processing circuit, an iterative closest point to minimize a difference between the first cloud of points and the second cloud of points based on the camera pose initial value to estimate the camera pose. ### Claim 4 (depends on 3) The 3D model reconstruction method of claim 3, wherein the camera pose comprises a rotation matrix and a translation vector indicating a movement of the depth camera between the first time point and the second time point. ### Claim 5 (depends on 4) The 3D model reconstruction method of claim 4, further comprising: applying, by the processing circuit, a volumetric integration based on the rotation matrix and the translation vector of the camera pose to update the 3D model. ### Claim 6 (depends on 4) The 3D model reconstruction method of claim 4, further comprising: building, by the processing circuit, a surface of the 3D model of the target object based on the rotation matrix and the translation vector of the camera pose. ### Claim 7 An electronic device, comprising: a processing circuit; a depth camera electrically connected to the processing circuit; one or more light sensors electrically connected to the processing circuit and arranged on the electronic device; a memory electrically connected to the processing circuit; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processing circuit, the one or more programs comprising instructions for: controlling the depth camera to capture a depth data of a target object when the electronic device moves around the target object; obtaining a camera pose initial value of the depth camera based on one or more laser beams detected by the one or more light sensors; and performing a 3D reconstruction of the target object using the depth data based on the camera pose initial value to output a 3D model of the target object, wherein to obtain the camera pose initial value of the depth camera comprises: receiving a tracker pose of the electronic device from the processing circuit, wherein the tracker pose is obtained according to the one or more laser beams; and obtaining the camera pose initial value according to the tracker pose, wherein the tracker pose is transferred from tracker coordinate to camera coordinate to obtain the camera pose initial value of the depth camera. ### Claim 8 (depends on 7) The electronic device as claimed in claim 7, wherein the one or more programs further comprise instructions for: estimating a camera pose between a first time point and a second time point based on the depth data at a first time point, the depth data at a second time point, and the camera pose initial value. ### Claim 9 (depends on 8) The electronic device as claimed in claim 8, wherein the one or more programs further comprise instructions for: converting the depth data obtained at the first time point to a first cloud of points by applying a depth map conversion; converting the depth data obtained at the second time point to a second cloud of points by applying the depth map conversion; and performing an iterative closest point to minimize a difference between the first cloud of points and the second cloud of points based on the camera pose initial value to estimate the camera pose. ### Claim 10 (depends on 9) The electronic device as claimed in claim 9, wherein the camera pose comprises a rotation matrix and a translation vector indicating a movement of the depth camera between the first time point and the second time point. ### Claim 11 (depends on 10) The electronic device as claimed in claim 10, wherein the one or more programs further comprise instructions for: applying a volumetric integration based on the rotation matrix and the translation vector of the camera pose to update the 3D model. ### Claim 12 (depends on 10) The electronic device as claimed in claim 10, wherein the one or more programs further comprise instructions for: building a surface of the 3D model of the target object based on the rotation matrix and the translation vector of the camera pose. ### Claim 13 A non-transitory computer readable storage medium storing one or more programs, comprising instructions, which when executed, causes a processing circuit to perform operations comprising: controlling a depth camera on an electronic device to capture a depth data of a target object when the electronic device moves around the target object; obtaining a camera pose initial value of the depth camera based on one or more laser beams detected by one or more light sensors on the electronic device; and performing a 3D reconstruction of the target object using the depth data based on the camera pose initial value to output a 3D model of the target object, wherein to obtain the camera pose initial value of the depth camera comprises: receiving a tracker pose of the electronic device from the processing circuit, wherein the tracker pose is obtained according to the one or more laser beams detected by the one or more light sensors; and obtaining the camera pose initial value according to the tracker pose, wherein the tracker pose is transferred from tracker coordinate to camera coordinate to obtain the camera pose initial value of the depth camera. ### Claim 14 (depends on 13) The non-transitory computer readable storage medium as claimed in claim 13, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: estimating a camera pose between a first time point and a second time point based on the depth data at a first time point, the depth data at a second time point, and the camera pose initial value. ### Claim 15 (depends on 14) The non-transitory computer readable storage medium as claimed in claim 14, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: converting the depth data obtained at the first time point to a first cloud of points by applying a depth map conversion; converting the depth data obtained at the second time point to a second cloud of points by applying the depth map conversion; and performing an iterative closest point to minimize a difference between the first cloud of points and the second cloud of points based on the camera pose initial value to estimate the camera pose. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium as claimed in claim 15, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: applying a volumetric integration based on a rotation matrix and a translation vector of the camera pose to update the 3D model. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium as claimed in claim 15, further comprising instructions, which when executed, causes the processing circuit to further perform operations comprising: building a surface of the 3D model of the target object based on a rotation matrix and a translation vector of the camera pose.