# US 11,308,347 B2 · Method of determining a similarity transformation between first and second coordinates of 3D features > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,347 B2 - Document ID: US11308347B2 - Status: Granted - Kind: B2 - Application number: 17/002,137 - Filed: 2020-08-25 - Published: 2020-12-10 - Granted: 2022-04-19 - Earliest claimed: 2019-08-02 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Daniel Ulbricht; Thomas Olszamowski - Web page: https://patentgrep.com/patent/US11308347B2 ## CPC - G06T 7/33: using feature-based methods - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/579: from motion - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06T 2200/04: involving 3D image data - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30244: Camera pose - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The invention is related to a method of determining a similarity transformation between first coordinates and second coordinates of 3D features, comprising providing a first plurality of 3D features having first coordinates in a first coordinate system which is associated with a first geometrical model of a first real object, wherein the first plurality of 3D features describes physical 3D features of the first real object, providing a second coordinate system, providing image information associated with a plurality of images captured by at least one camera, for each respective 3D feature of at least part of the first plurality of 3D features, wherein the respective 3D feature is captured by at least two of the plurality of images, determining camera poses of the at least one camera in the second coordinate system while the at least two of the plurality of images are captured, determining for the respective 3D feature a second coordinate in the second coordinate system according to the at least two of the plurality of images and the camera poses, and the method further comprising determining a similarity transformation between the first coordinates and the second coordinates of the at least part of the first plurality of 3D features, wherein the similarity transformation includes at least one translation, at least one rotation, at least one scale and/or their combinations in 3D space. ## Claims ### Claim 1 A non-transitory computer readable medium comprising computer readable code executable by one or more processors to: obtain an image of a real object captured by a camera, wherein the real object comprises a 3D feature; obtain a first geometrical model of the real object, wherein the 3D feature has first coordinates in the first geometrical model; determine a camera pose for the camera based on the image; determine at least second coordinates for the 3D feature according to the determined camera pose; and determine at least one scale value according to the first coordinates and the at least second coordinates. ### Claim 2 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the scale value comprises a scale of the first geometrical model compared to the real object. ### Claim 3 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the computer readable code to determine the at least one scale value further comprises computer readable code to: determine a similarity transformation between the first coordinates and the at least second coordinates, wherein the scale value is determined based on the similarity transformation. ### Claim 4 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the first geometrical model is associated with a first coordinate system, and wherein the at least second coordinates are associated with a second coordinate system. ### Claim 5 (depends on 1) The non-transitory computer readable medium of claim 1, wherein the first coordinates and the second coordinates are in a common coordinate system. ### Claim 6 (depends on 1) The non-transitory computer readable medium of claim 1, further comprising computer readable code to: provide a second plurality of 3D features having third coordinates in an additional coordinate system, wherein the additional coordinate system is associated with a second geometrical model of a second real object, and wherein the second plurality of 3D features describes physical 3D features of the second real object, wherein for each respective 3D feature of the second plurality of 3D features, at least one 3D feature of the second plurality of 3D features is captured in each of the plurality of images. ### Claim 7 (depends on 1) The computer readable medium of claim 1, further comprising computer readable code to: obtain information from a tracking system; and determine the camera pose of the camera in a third coordinate system associated with the tracking system. ### Claim 8 A system, comprising: one or more processors; and one or more non-transitory computer readable media comprising computer readable code executable by the one or more processors to: obtain an image of a real object captured by a camera, wherein the real object comprises a 3D feature; obtain a first geometrical model of the real object, wherein the 3D feature has first coordinates in the first geometrical model; determine a camera pose for the camera based on the image; determine at least second coordinates for the 3D feature according to the determined camera pose; and determine at least one scale value according to the first coordinates and the at least second coordinates. ### Claim 9 (depends on 8) The system of claim 8, wherein the scale value comprises a scale of the first geometrical model compared to the real object. ### Claim 10 (depends on 8) The system of claim 8, wherein the computer readable code to determine the at least one scale value further comprises computer readable code to: determine a similarity transformation between the first coordinates and the at least second coordinates, wherein the scale value is determined based on the similarity transformation. ### Claim 11 (depends on 8) The system of claim 8, wherein the first geometrical model is associated with a first coordinate system, and wherein the at least second coordinates are associated with a second coordinate system. ### Claim 12 (depends on 8) The system of claim 8, wherein the first coordinates and the second coordinates are in a common coordinate system. ### Claim 13 (depends on 8) The system of claim 8, further comprising computer readable code to: provide a second plurality of 3D features having third coordinates in an additional coordinate system, wherein the additional coordinate system is associated with a second geometrical model of a second real object, and wherein the second plurality of 3D features describes physical 3D features of the second real object, wherein for each respective 3D feature of the second plurality of 3D features, at least one 3D feature of the second plurality of 3D features is captured in each of the plurality of images. ### Claim 14 (depends on 8) The system of claim 8, further comprising computer readable code to: obtain information from a tracking system; and determine the camera pose of the camera in a third coordinate system associated with the tracking system. ### Claim 15 A method, comprising: obtain an image of a real object captured by a camera, wherein the real object comprises a 3D feature; obtain a first geometrical model of the real object, wherein the 3D feature has first coordinates in the first geometrical model; determine a camera pose for the camera based on the image; determine at least second coordinates for the 3D feature according to the determined camera pose; and determine at least one scale value according to the first coordinates and the at least second coordinates. ### Claim 16 (depends on 15) The method of claim 15, wherein the scale value comprises a scale of the first geometrical model compared to the real object. ### Claim 17 (depends on 15) The method of claim 15, wherein determining the at least one scale value further comprises: determining a similarity transformation between the first coordinates and the at least second coordinates, wherein the scale value is determined based on the similarity transformation. ### Claim 18 (depends on 15) The method of claim 15, wherein the first geometrical model is associated with a first coordinate system, and wherein the at least second coordinates are associated with a second coordinate system. ### Claim 19 (depends on 15) The method of claim 15, wherein the first coordinates and the second coordinates are in a common coordinate system. ### Claim 20 (depends on 15) The method of claim 15, further comprising: providing a second plurality of 3D features having third coordinates in an additional coordinate system, wherein the additional coordinate system is associated with a second geometrical model of a second real object, and wherein the second plurality of 3D features describes physical 3D features of the second real object, wherein for each respective 3D feature of the first plurality of 3D features, at least one 3D feature of the second plurality of 3D features is captured in each of the plurality of images.