# US 2022/0036660 A1 · Surface aware lens > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0036660 A1 - Document ID: US20220036660A1 - Status: Published application - Kind: A1 - Application number: 17/505,370 - Filed: 2021-10-19 - Published: 2022-02-03 - Earliest claimed: 2017-01-09 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Andrew James McPhee; Ebony James Charlton; Samuel Edward Hare; Michael John Evans; Jokubas Dragis; Ricardo Sanchez-Saez - Web page: https://patentgrep.com/patent/US20220036660A1 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06T 15/205: {Image-based rendering} - G06T 15/60: Shadow generation - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering three-dimensional virtual objects within real-world environments. Virtual rendering of a three-dimensional virtual object can be altered appropriately as a user moves around the object in the real-world through utilization of a redundant tracking system comprising multiple tracking sub-systems. Virtual object rendering can be with respect to a reference surface in a real-world three-dimensional space depicted in a camera view of a mobile computing device. ## Claims ### Claim 1 A system comprising: a memory; a redundant tracking subsystem comprising a set of tracking subsystems, the set of tracking subsystems comprising: a gyroscopic tracking system, a natural feature tracking system, and simultaneous localization and mapping tracking; and at least one hardware processor coupled to perform operations comprising: assigning a virtual object to a position in three-dimensional space captured within a camera view; rendering the virtual object to appear at the position in the three-dimensional space within the camera view; and maintaining, via the redundant tracking subsystem, the virtual object rendered at the position in the three-dimensional space. ### Claim 2 (depends on 1) The system of claim 1, wherein the rendering of the virtual object includes rendering a separate virtual representation related to the virtual object. ### Claim 3 (depends on 2) The system of claim 2, wherein the separate virtual representation is a simulated shadow of the virtual object. ### Claim 4 (depends on 1) The system of claim 1, further comprising receiving input indicative of a selection of a template from among a set of templates, the template corresponding to the virtual object. ### Claim 5 (depends on 1) The system of claim 1, wherein the virtual object comprises customized personal avatar content. ### Claim 6 (depends on 1) The system of claim 1, wherein the virtual object includes content selected based on external factors related to the three-dimensional space. ### Claim 7 (depends on 1) The system of claim 1, wherein the operations further comprise identifying tracking indicia operable to track the virtual object in the three-dimensional space, wherein the maintaining of the virtual object rendered at the position in the three-dimensional space comprises tracking the virtual object using the tracking indicia. ### Claim 8 (depends on 7) The system of claim 7, wherein: the tracking of the virtual object using the tracking indicia comprises tracking, by a first tracking subsystem from among the set of tracking subsystems, the virtual object at the position in the three-dimensional space using the tracking indicia; the maintaining of the virtual object rendered at the position in the three-dimensional space further comprises: detecting, by the first tracking subsystem, an interruption of the tracking indicia; and in response to detecting the interruption of the tracking indicia, tracking the virtual object at the position in the three-dimensional space via a second tracking subsystem from among the set of tracking subsystems. ### Claim 9 (depends on 1) The system of claim 1, wherein the operations further comprise detecting a reference surface in a three-dimensional space captured within a camera view, wherein the assigning of the virtual object to the position in three-dimensional space is based on the reference surface. ### Claim 10 (depends on 9) The system of claim 9, wherein the detecting of the reference surface is based on user input. ### Claim 11 (depends on 9) The system of claim 9, wherein the position in the three-dimensional space is a predefined distance from a reference surface. ### Claim 12 (depends on 11) The system of claim 11, wherein maintaining the virtual object at the position in the three-dimensional space comprises maintaining the virtual object rendered at the predefined distance from the reference surface. ### Claim 13 A method comprising: assigning a virtual object to a position in three-dimensional space captured within a camera view; rendering the virtual object to appear at the position in the three-dimensional space within the camera view; and maintaining, via a redundant tracking subsystem, the virtual object rendered at the position in the three-dimensional space, the redundant tracking subsystem comprising a set of tracking subsystems, the set of tracking subsystems comprising: a gyroscopic tracking system, a natural feature tracking system, and simultaneous localization and mapping tracking. ### Claim 14 (depends on 13) The method of claim 13, wherein the rendering of the virtual object includes rendering a separate virtual representation related to the virtual object. ### Claim 15 (depends on 13) The method of claim 13, wherein the virtual object includes content selected based on external factors related to the three-dimensional space. ### Claim 16 (depends on 13) The method of claim 13, further comprising identifying tracking indicia operable to track the virtual object in the three-dimensional space, wherein the maintaining of the virtual object rendered at the position in the three-dimensional space comprises tracking the virtual object using the tracking indicia. ### Claim 17 (depends on 16) The method of claim 16, wherein: the tracking of the virtual object using the tracking indicia comprises tracking, by a first tracking subsystem from among the set of tracking subsystems, the virtual object at the position in the three-dimensional space using the tracking indicia; the maintaining of the virtual object rendered at the position in the three-dimensional space further comprises: detecting, by the first tracking subsystem, an interruption of the tracking indicia; and in response to detecting the interruption of the tracking indicia, tracking the virtual object at the position in the three-dimensional space via a second tracking subsystem from among the set of tracking subsystems. ### Claim 18 (depends on 13) The method of claim 13, further comprising detecting a reference surface in a three-dimensional space captured within a camera view, wherein the assigning of the virtual object to the position in three-dimensional space is based on the reference surface. ### Claim 19 (depends on 18) The method of claim 18, wherein the position in the three-dimensional space is a predefined distance from a reference surface, wherein maintaining the virtual object at the position in the three-dimensional space comprises maintaining the virtual object rendered at the predefined distance from the reference surface. ### Claim 20 A non-transitory machine-readable storage medium that includes instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising: assigning a virtual object to a position in three-dimensional space captured within a camera view; rendering the virtual object to appear at the position in the three-dimensional space within the camera view; and maintaining, via a redundant tracking subsystem, the virtual object rendered at the position in the three-dimensional space, the redundant tracking subsystem comprising a set of tracking subsystems, the set of tracking subsystems comprising: a gyroscopic tracking system, a natural feature tracking system, and simultaneous localization and mapping tracking.