# US 10,311,643 B2 · Accurate positioning of augmented reality content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,643 B2 - Document ID: US10311643B2 - Status: Granted - Kind: B2 - Application number: 15/592,047 - Filed: 2017-05-10 - Published: 2018-01-04 - Granted: 2019-06-04 - Earliest claimed: 2014-11-11 - Assignees: YouAR INC. - Applicants: YouAR INC. - Inventors: Oliver Clayton Daniels; David Morris Daniels; Raymond Victor Di Carlo; Carlo J. Calica; Luke Timothy Hartwig - Web page: https://patentgrep.com/patent/US10311643B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G01S 19/01: Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO - G06V 20/20: in augmented reality scenes - G06T 2215/16: Using real world measurements to influence rendering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A system for accurately positioning augmented reality (AR) content within a coordinate system such as the World Geodetic System (WGS) may include AR content tethered to trackable physical features. As the system is used by mobile computing devices, each mobile device may calculate and compare relative positioning data between the trackable features. The system may connect and group the trackable features hierarchically, as measurements are obtained. As additional measurements are made of the trackable features in a group, the relative position data may be improved, e.g., using statistical methods. ## Claims ### Claim 1 A computer-implemented method for accurately locating augmented reality (AR) content, the method comprising: measuring, using one or more sensors of a computing device, a first six-degree-of-freedom (DOF) vector between a first vantage point of the computing device and a first trackable feature in a first sensor range of the computing device; measuring, using the one or more sensors of the computing device, a second six-DOF vector between a second vantage point of the computing device and a second trackable feature in a second sensor range of the computing device; measuring, using the one or more sensors of the computing device, a third six-DOF vector between the first vantage point and the second vantage point; estimating, based on the first vector, the second vector, and the third vector, using a processor of the computing device, a fourth six-DOF vector between the first trackable feature and the second trackable feature; storing the estimated fourth vector in a data store in association with the first trackable feature and the second trackable feature, such that the estimated fourth vector represents a spatial relationship between the first trackable feature and the second trackable feature; determining, within a common spatial coordinate system, a first set of spatial coordinates for the first trackable feature and a second set of spatial coordinates for the second trackable feature; storing, in the data store, the first set of spatial coordinates associated with the first trackable feature and the second set of spatial coordinates associated with the second trackable feature, such that the first and second sets of spatial coordinates represent respective locations of the first and second trackable features within the common coordinate system; wherein the data store includes a fifth six-DOF vector representing a spatial relationship and an orientation of an AR content item relative to the first trackable feature; associating the first trackable feature and the second trackable feature with a first group of trackable features in the data store, each trackable feature in the first group having a position and orientation defined with respect to a first coordinate system; measuring, using the computing device, a seventh six-DOF vector between the first trackable feature and a third trackable feature associated with a second group of trackable features in the data store, each trackable feature in the second group having a position and orientation defined with respect to a second coordinate system; and calculating a positional offset and an orientation offset between the first group and the second group based on the seventh six-DOF vector. ### Claim 2 (depends on 1) The method of claim 1, further comprising: measuring, using the one or more sensors of the computing device, a sixth six-DOF vector between a current vantage point of the computing device and the second trackable feature when the second trackable feature is in a current sensor range of the computing device; determining, using the processor of the computing device and based on the fourth vector, the fifth vector, and the sixth vector, a six-DOF current rendering vector for the current vantage point of the computing device, the current rendering vector including an orientation component; and presenting, at a graphical display of the computing device, an AR representation including the AR content item rendered based on the current rendering vector within a live view of a real-world environment for a current field of view (FOV) of the computing device. ### Claim 3 (depends on 2) The method of claim 2, wherein the first trackable feature is outside the current sensor range of the computing device. ### Claim 4 (depends on 1) The method of claim 1, further comprising assigning an uncertainty value to the estimated fourth vector in the data store, based on respective uncertainties in the measurements of the first vector, the second vector, and the third vector. ### Claim 5 (depends on 1) The method of claim 1, wherein the computing device is a first computing device, the method further comprising: updating at least one component of the fourth vector based on a measurement by a second computing device. ### Claim 6 (depends on 5) The method of claim 5, wherein updating the at least one component of the fourth vector comprises updating an uncertainty value assigned to the fourth vector. ### Claim 7 (depends on 1) The method of claim 1, further comprising: generating a third group comprising the first, second, and third trackable features, and defining a respective position and orientation of each of the first, second, and third trackable features with respect to a third coordinate system. ### Claim 8 (depends on 1) The method of claim 1, wherein the computing device is a first computing device, the method further comprising: updating at least one component of the fourth vector based on a measurement by a second computing device, wherein updating the estimated fourth vector is performed independently with respect to the seventh vector. ### Claim 9 (depends on 1) The method of claim 1, further comprising: adjusting the positional offset and the orientation offset between the first group and the second group based on at least one additional measurement between a selected trackable feature of the first group and a selected trackable feature of the second group. ### Claim 10 (depends on 1) The method of claim 1, further comprising associating an uncertainty value to the positional offset, the uncertainty value being based on an uncertainty of the seventh six-DOF vector; and adjusting the uncertainty value of the positional offset based on at least one additional measurement between a selected trackable feature of the first group and a selected trackable feature of the second group. ### Claim 11 (depends on 1) The method of claim 1, wherein the common spatial coordinate system comprises the World Geodetic System. ### Claim 12 (depends on 1) The method of claim 1, wherein the computing device is a first computing device, the method further comprising: measuring, using one or more sensors of a second computing device, an eighth six-degree-of-freedom (DOF) vector between a vantage point of the second computing device and the first trackable feature; and updating the first set of spatial coordinates based on the eighth vector. ### Claim 13 (depends on 12) The method of claim 12, wherein updating the first set of spatial coordinates comprises updating an uncertainty associated with the first set of spatial coordinates. ### Claim 14 (depends on 1) The method of claim 1, wherein the data store is at least partially disposed on a communications network, and the computing device is in communication with the data store via the communications network. ### Claim 15 A computer-implemented method, comprising: maintaining, in a data store, a set of trackable points, each trackable point representing a trackable physical feature to which an augmented reality (AR) content item is tetherable; for each trackable point in the set of trackable points, maintaining a first set of estimated spatial values and a second set of estimated spatial values in the data store, the first set of estimated spatial values representing a position of the respective trackable point in a respective group coordinate system, and, by extension, one or more spatial relationships between the respective trackable point and all other trackable points of a respective group of trackable points positioned within the respective group coordinate system, and the second set of estimated spatial values representing a position of the respective trackable point in a respective global coordinate system; obtaining first measurement data using at least one first mobile computing device, the first measurement data comprising a plurality of first measurements relating to a plurality of onsite trackable physical features within a selected real-world environment, the at least one first mobile computing device being onsite with respect to the selected real-world environment; for each first set of spatial values corresponding to the plurality of onsite trackable physical features, maintaining a respective uncertainty value corresponding to the first measurement data; obtaining second measurement data using at least one onsite second computing device, the second measurement data comprising at least one second measurement relating to the plurality of onsite trackable physical features within the selected real-world environment; and updating the first and second estimated spatial values of at least one trackable point in the set of trackable points, based on the second measurement data. ### Claim 16 (depends on 15) The method of claim 15, wherein updating the first and second estimated spatial values of the at least one trackable point in the set of trackable points includes reducing the uncertainty value for the first set of spatial values. ### Claim 17 (depends on 15) The method of claim 15, wherein at least one of the respective group coordinate systems has an origin and an orientation. ### Claim 18 (depends on 15) The method of claim 15, further comprising: forming a parent group comprising at least two of the respective groups of trackable points, wherein each of the at least two groups is a respective child group having a third set of spatial values representing a position and orientation of the respective group coordinate system with respect to the parent group. ### Claim 19 (depends on 18) The method of claim 18, further comprising: for the trackable points in the at least two child groups, calculating and assigning the first set of spatial values in relation to the parent group coordinate system; and removing the at least two child groups from the system. ### Claim 20 (depends on 15) The method of claim 15, further comprising: presenting, at a graphical display of the at least one second computing device, an AR representation including an AR content item tethered to one of the trackable physical features within a live view of the selected real-world environment. ### Claim 21 (depends on 20) The method of claim 20, wherein the trackable physical feature to which the AR content item is tethered is untracked by the first computing device. ### Claim 22 (depends on 21) The method of claim 21, wherein the trackable physical feature to which the AR content item is tethered is out of sensing range of the second computing device. ### Claim 23 (depends on 15) The method of claim 15, further comprising generating a new trackable point in the set of trackable points; and adding the new trackable point to one of the existing groups of trackable points, based on third measurement data taken in the selected real-world environment, such that a respective spatial relationship is defined between the new trackable point and at least one of the other trackable points. ### Claim 24 (depends on 23) The method of claim 23, further comprising: maintaining the first set of estimated spatial values and the second set of estimated spatial values for the new trackable point in the data store, the first set of estimated spatial values representing the spatial relationship between the new trackable point and at least one of the other trackable points, the second set of spatial values representing the position of the new trackable point in the global coordinate system. ### Claim 25 A computer-implemented method for accurately locating augmented reality (AR) content, the method comprising: measuring, using one or more sensors of a first computing device, a first six-degree-of-freedom (DOF) vector between a first vantage point of the first computing device and a first trackable feature in a first sensor range of the first computing device; measuring, using the one or more sensors of the first computing device, a second six-DOF vector between a second vantage point of the first computing device and a second trackable feature in a second sensor range of the first computing device; measuring, using the one or more sensors of the first computing device, a third six-DOF vector between the first vantage point and the second vantage point; estimating, based on the first vector, the second vector, and the third vector, using a processor of the first computing device, a fourth six-DOF vector between the first trackable feature and the second trackable feature; storing the estimated fourth vector in a data store in association with the first trackable feature and the second trackable feature, such that the estimated fourth vector represents a spatial relationship between the first trackable feature and the second trackable feature; determining, within a common spatial coordinate system, a first set of spatial coordinates for the first trackable feature and a second set of spatial coordinates for the second trackable feature; and storing, in the data store, the first set of spatial coordinates associated with the first trackable feature and the second set of spatial coordinates associated with the second trackable feature, such that the first and second sets of spatial coordinates represent respective locations of the first and second trackable features within the common coordinate system; wherein the data store includes a fifth six-DOF vector representing a spatial relationship and an orientation of an AR content item relative to the first trackable feature; measuring, using one or more sensors of a second computing device, a sixth six-degree-of-freedom (DOF) vector between a vantage point of the second computing device and the first trackable feature; and updating the first set of spatial coordinates based on the sixth vector. ### Claim 26 (depends on 25) The method of claim 25, further comprising: measuring, using the one or more sensors of the first computing device, a seventh six-DOF vector between a current vantage point of the first computing device and the second trackable feature when the second trackable feature is in a current sensor range of the first computing device; determining, using the processor of the first computing device and based on the fourth vector, the fifth vector, and the seventh vector, a six-DOF current rendering vector for the current vantage point of the first computing device, the current rendering vector including an orientation component; and presenting, at a graphical display of the first computing device, an AR representation including the AR content item rendered based on the current rendering vector within a live view of a real-world environment for a current field of view (FOV) of the first computing device. ### Claim 27 (depends on 26) The method of claim 26, wherein the first trackable feature is outside the current sensor range of the first computing device. ### Claim 28 (depends on 25) The method of claim 25, further comprising assigning an uncertainty value to the estimated fourth vector in the data store, based on respective uncertainties in the measurements of the first vector, the second vector, and the third vector. ### Claim 29 (depends on 25) The method of claim 25, further comprising: updating at least one component of the fourth vector based on a measurement by the second computing device. ### Claim 30 (depends on 29) The method of claim 29, wherein updating the at least one component of the fourth vector comprises updating an uncertainty value assigned to the fourth vector. ### Claim 31 (depends on 25) The method of claim 25, further comprising: associating the first trackable feature and the second trackable feature with a first group of trackable features in the data store, each trackable feature in the first group having a position and orientation defined with respect to a first coordinate system; measuring, using the first computing device, an eighth six-DOF vector between the first trackable feature and a third trackable feature associated with a second group of trackable features in the data store, each trackable feature in the second group having a position and orientation defined with respect to a second coordinate system; and calculating a positional offset and an orientation offset between the first group and the second group based on the eighth six-DOF vector. ### Claim 32 (depends on 31) The method of claim 31, further comprising: generating a third group comprising the first, second, and third trackable features, and defining a respective position and orientation of each of the first, second, and third trackable features with respect to a third coordinate system. ### Claim 33 (depends on 31) The method of claim 31, further comprising: updating at least one component of the fourth vector based on a measurement by the second computing device, wherein updating the estimated fourth vector is performed independently with respect to the eighth vector. ### Claim 34 (depends on 31) The method of claim 31, further comprising: adjusting the positional offset and the orientation offset between the first group and the second group based on at least one additional measurement between a selected trackable feature of the first group and a selected trackable feature of the second group. ### Claim 35 (depends on 31) The method of claim 31, further comprising associating an uncertainty value to the positional offset, the uncertainty value being based on an uncertainty of the eighth six-DOF vector; and adjusting the uncertainty value of the positional offset based on at least one additional measurement between a selected trackable feature of the first group and a selected trackable feature of the second group. ### Claim 36 (depends on 25) The method of claim 25, wherein the common spatial coordinate system comprises the World Geodetic System. ### Claim 37 (depends on 25) The method of claim 25, wherein updating the first set of spatial coordinates comprises updating an uncertainty associated with the first set of spatial coordinates. ### Claim 38 (depends on 25) The method of claim 25, wherein the data store is at least partially disposed on a communications network, and the first computing device is in communication with the data store via the communications network.