# US 10,319,141 B2 · Method and system for vision based 3D reconstruction and object tracking > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,319,141 B2 - Document ID: US10319141B2 - Status: Granted - Kind: B2 - Application number: 15/188,223 - Filed: 2016-06-21 - Published: 2017-12-21 - Granted: 2019-06-11 - Earliest claimed: 2016-06-21 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Lejing Wang - Web page: https://patentgrep.com/patent/US10319141B2 ## CPC - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 7/593: from stereo images - G06T 7/85: {Stereo camera calibration} - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Three dimensional model construction includes obtaining a first image of an object captured by a first camera facing a first direction, obtaining a second image of the object captured by a second camera facing a second direction, wherein the second camera captures the second image of the object as a reflection in a mirror, determining a virtual baseline distance between the first camera and a device including the second camera based on a location of the first camera, a known spatial relationship between the first camera and the device, and a reflected distance of the second camera, and generating at least a partial 3D model of the first object using the first image, the second image, and the virtual baseline distance. ## Claims ### Claim 1 A method for 3D model construction, comprising: obtaining a first image of an object captured by a first camera of an electronic device, the first camera facing a first direction; obtaining a second image of the object captured by a second camera of the device, the second camera facing a second direction, wherein the second camera captures the second image of the object as a reflection in a mirror in front of the second camera; determining a virtual baseline distance between the first camera and the second camera based on an apparent distance of the first camera to the second camera as reflected in the mirror based on a reflected distance of at least part of the electronic device in the second image and a known spatial relationship between the first camera and the at least part of the electronic device; and generating at least a partial 3D model of the first object using the first image, the second image, and the virtual baseline distance. ### Claim 2 (depends on 1) The method of claim 1, wherein the first camera faces an object from a first surface of the electronic device, and wherein the second camera faces the mirror from an opposite surface from the first surface of the electronic device. ### Claim 3 (depends on 1) The method of claim 1, wherein determining a virtual baseline distance between the first camera and the second camera comprises: identifying the at least part of the electronic device in the second image, determining a distance between the electronic device and a mirrored position of the second camera based on characteristics of the electronic device in the second image, and determining the virtual baseline based on the determined distance between the electronic device and the mirrored position of the second camera. ### Claim 4 (depends on 1) The method of claim 1, further comprising: initializing a tracking mechanism using the first image, the second image, and the virtual baseline distance. ### Claim 5 (depends on 4) The method of claim 4, further comprising: capturing a third image by one of the first camera and the second camera, wherein the third image comprises at least part of the object; and updating the 3D model using the third image and a tracked camera pose of the one of the first camera and the second camera. ### Claim 6 (depends on 5) The method of claim 5, further comprising: determining a tracked position of the device according to the third image, wherein updating the 3D model comprises using the third image and the tracked position of the device according to the third image. ### Claim 7 (depends on 1) The method of claim 1, wherein generating at least a partial 3D model of the object comprises generating a true-to-scale model of the object. ### Claim 8 A computer readable medium comprising computer readable code executable by one or more processors to: obtain a first image of an object captured by a first camera of an electronic device, the first camera facing a first direction; obtain a second image of the object captured by a second camera of the electronic device, the second camera facing a second direction, wherein the second camera captures the second image of the object as a reflection in a mirror; determine a virtual baseline distance between the first camera and the second camera based on an apparent distance of the first camera to the second camera as reflected in the mirror based on a reflected distance of at least part of the electronic device in the second image and a known spatial relationship between the first camera and the at least part of the electronic device; and generate at least a partial 3D model of the object using the first image, the second image, and the virtual baseline distance. ### Claim 9 (depends on 8) The computer readable medium of claim 8, wherein the first camera faces the object from a first surface of the electronic device, and wherein the second camera faces the mirror from an opposite surface from the first surface of the electronic device. ### Claim 10 (depends on 8) The computer readable medium of claim 8, wherein determining a virtual baseline distance between the first camera and the second camera comprises: identifying the at least part of the electronic device in the second image, determining a distance between the electronic device and a mirror based on characteristics of the electronic device in the second image, and determining the virtual baseline based on the determined distance between the electronic device and the mirror. ### Claim 11 (depends on 8) The computer readable medium of claim 8, further comprising computer readable code executable by one or more processors to: initialize a tracking mechanism using the first image, the second image, and the virtual baseline distance. ### Claim 12 (depends on 11) The computer readable medium of claim 11, further comprising computer readable code executable by one or more processors to: capture a third image by one of the first camera and the second camera, wherein the third image comprises at least part of the object; and update the 3D model using the third image and a tracked camera pose of the one of the first camera and the second camera. ### Claim 13 (depends on 12) The computer readable medium of claim 12, further comprising computer readable code executable by one or more processors to: determine a tracked position of the device according to the third image, wherein the computer readable code to update the 3D model comprises computer readable code to use the third image and the tracked position of the device according to the third image. ### Claim 14 (depends on 8) The computer readable medium of claim 8, wherein the computer readable code to generate at least a partial 3D model of the object comprises computer readable code to generate a true-to-scale model of the object. ### Claim 15 A system for 3D model construction, comprising: a first camera of an electronic device directed in a first direction; a second camera of the electronic device directed in a second direction; one or more processors; and a memory coupled to the one or more processors and comprising computer readable code executable by the one or more processors to: obtain a first image of an object captured by the first camera; obtain a second image of the object captured by the second camera as a reflection in a mirror in front of the first camera; determine a virtual baseline distance between the first camera and the second camera based on an apparent distance of the first camera to the second camera as reflected in the mirror based on a reflected distance of at least part of the electronic device in the second image and a known spatial relationship between the first camera and the at least part of the electronic device; and generate at least a partial 3D model of the first object using the first image, the second image, and the virtual baseline distance. ### Claim 16 (depends on 15) The system of claim 15, wherein the first camera faces an object from a first surface of the electronic device, and wherein the second camera faces the mirror from an opposite surface from the first surface of the electronic device. ### Claim 17 (depends on 15) The system of claim 15, wherein the computer readable code to determine a virtual baseline distance between the first camera and the second camera comprises computer readable code to: identify the at least part of the electronic device in the second image, determine a distance between the electronic device and the mirror based on characteristics of the electronic device in the second image, and determine the virtual baseline based on the determined distance between the electronic device and the mirror. ### Claim 18 (depends on 15) The system of claim 15, further comprising computer readable code executable by one or more processors to: initialize a tracking mechanism using the first image, the second image, and the virtual baseline distance. ### Claim 19 (depends on 15) The system of claim 15, further comprising computer readable code executable by one or more processors to: capture a third image by one of the first camera and the second camera, wherein the third image comprises at least part of the object; and update the 3D model using the third image and a tracked camera pose of the one of the first camera and the second camera. ### Claim 20 (depends on 15) The system of claim 15, wherein the computer readable code to generate at least a partial 3D model of the object comprises computer readable code to generate a true-to-scale model of the object.