# US 12,183,310 B2 · Depth based foveated rendering for display systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,310 B2 - Document ID: US12183310B2 - Status: Granted - Kind: B2 - Application number: 18/330,989 - Filed: 2023-06-07 - Published: 2023-10-05 - Granted: 2024-12-31 - Earliest claimed: 2018-03-16 - Assignees: Magic Leap, Inc. - Applicants: Magic Leap, Inc. - Inventors: Vaibhav Mathur; Lionel Ernest Edwin; Xiaoyang Zhang; Bjorn Nicolaas Servatius Vlaskamp - Web page: https://patentgrep.com/patent/US12183310B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G09G 5/391: Resolution modifying circuits, e.g. variable screen formats - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/147: using display panels - G06T 5/70: Denoising; Smoothing - G06T 15/00: Three-dimensional [3D] image rendering - G02B 2027/0147: {comprising a device modifying the resolution of the displayed image} - G06T 2210/36: Level of detail - G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen - G09G 2320/0606: Manual adjustment - G09G 2320/0613: The adjustment depending on the type of the information to be displayed - G09G 2340/04: Changes in size, position or resolution of an image - G09G 2340/0428: Gradation resolution change - G09G 2340/12: Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels - G09G 2354/00: Aspects of interface with display user - G09G 2360/144: the light being ambient light - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Methods and systems for depth-based foveated rendering in the display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include determining a fixation point of a user's eyes. Location information associated with a first virtual object to be presented to the user via a display device is obtained. A resolution-modifying parameter of the first virtual object is obtained. A particular resolution at which to render the first virtual object is identified based on the location information and the resolution-modifying parameter of the first virtual object. The particular resolution is based on a resolution distribution specifying resolutions for corresponding distances from the fixation point. The first virtual object rendered at the identified resolution is presented to the user via the display system. ## Claims ### Claim 1 A wearable display system comprising: a display configured to output light at a plurality of depths to present virtual content to a user of the wearable display system; one or more processors communicatively coupled to the display; and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: determining a three-dimensional (3D) fixation point of eyes of the user; determining a 3D virtual object position at which a virtual object is to be presented within a field of view (FOV) of the user via the display; based at least partly on the fixation point and the virtual object position, determining a particular resolution at which to render the virtual object, wherein the FOV is divided into a plurality of zones that are arranged relative to the fixation point and that are each assigned an associated resolution, wherein different zones of the plurality of zones encompass different 3D volumes within the FOV, wherein at least two different zones that include a same lateral position and different depth positions are associated with different resolutions, and wherein the particular resolution is determined as the resolution that is associated with the particular zone that includes both a lateral component and a depth component of the virtual object position; and causing the display to present the virtual object at the virtual object position and at the particular resolution. ### Claim 2 (depends on 1) The wearable display system of claim 1, wherein the operations further comprise: determining a content type of the virtual object, wherein the particular resolution at which to render the virtual object is further based on the content type. ### Claim 3 (depends on 1) The wearable display system of claim 1, wherein the operations further comprise: adjusting the particular resolution of the virtual object based at least partly on input from the user. ### Claim 4 (depends on 1) The wearable display system of claim 1, wherein determining the particular resolution at which to render the virtual object further includes: determining that the virtual object is to be presented in at least two of the plurality of zones; and determining the particular resolution for the virtual object as the resolution that is associated with one of the at least two zones. ### Claim 5 (depends on 1) The wearable display system of claim 1, wherein the zone that includes the fixation point is associated with a maximum resolution supported by the wearable display system. ### Claim 6 (depends on 1) The wearable display system of claim 1, wherein the fixation point is in a zone that is approximately at a center of the FOV. ### Claim 7 (depends on 1) The wearable display system of claim 1, wherein the operations further comprise: determining a proximity of the virtual object to a boundary of one of the plurality of zones; and modifying a presentation of the virtual object based on the determined proximity. ### Claim 8 (depends on 7) The wearable display system of claim 7, wherein modifying the presentation of the virtual object includes adjusting at least one of the associated resolution, size, position, or geometry of one or more of the plurality of zones. ### Claim 9 (depends on 7) The wearable display system of claim 7, wherein modifying the presentation of the virtual object includes blurring the virtual object through application of a blurring process. ### Claim 10 A computer-implemented method comprising: determining a three-dimensional (3D) fixation point of eyes of a user of a wearable display system comprising a display configured to output light at a plurality of depths to present virtual content to the user; determining a 3D virtual object position at which a virtual object is to be presented within a field of view (FOV) of the user via the display; based at least partly on the fixation point and the virtual object position, determining a particular resolution at which to render the virtual object, wherein the FOV is divided into a plurality of zones that are arranged relative to the fixation point and that are each assigned an associated resolution, wherein different zones of the plurality of zones encompass different 3D volumes within the FOV, wherein at least two different zones that include a same lateral position and different depth positions are associated with different resolutions, and wherein the particular resolution is determined as the resolution that is associated with at least one particular zone that includes both a lateral component and a depth component of the virtual object position; and causing the display to present the virtual object at the virtual object position and at the particular resolution. ### Claim 11 (depends on 10) The method of claim 10, further comprising: determining a content type of the virtual object, wherein the particular resolution at which to render the virtual object is further based on the content type. ### Claim 12 (depends on 10) The method of claim 10, further comprising: adjusting the particular resolution of the virtual object based at least partly on input from the user. ### Claim 13 (depends on 10) The method of claim 10, wherein determining the particular resolution at which to render the virtual object further includes: determining that the virtual object is to be presented in at least two of the plurality of zones; and determining the particular resolution for the virtual object as the resolution that is associated with one of the at least two zones. ### Claim 14 (depends on 10) The method of claim 10, wherein the zone that includes the fixation point is associated with a maximum resolution supported by the wearable display system. ### Claim 15 (depends on 10) The method of claim 10, wherein the fixation point is in a zone that is approximately at a center of the FOV. ### Claim 16 (depends on 10) The method of claim 10, further comprising: determining a proximity of the virtual object to a boundary of one of the plurality of zones; and modifying a presentation of the virtual object based on the determined proximity. ### Claim 17 (depends on 16) The method of claim 16, wherein modifying the presentation of the virtual object includes adjusting at least one of the associated resolution, size, position, or geometry of one or more of the plurality of zones. ### Claim 18 (depends on 16) The method of claim 16, wherein modifying the presentation of the virtual object includes blurring the virtual object through application of a blurring process. ### Claim 19 Non-transitory computer storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: determining a three-dimensional (3D) fixation point of eyes of a user of a wearable display system comprising a display configured to output light at a plurality of depths to present virtual content to the user; determining a 3D virtual object position at which a virtual object is to be presented within a field of view (FOV) of the user via the display; based at least partly on the fixation point and the virtual object position, determining a particular resolution at which to render the virtual object, wherein the FOV is divided into a plurality of zones that are arranged relative to the fixation point and that are each assigned an associated resolution, wherein different zones of the plurality of zones encompass different 3D volumes within the FOV, wherein at least two different zones that include a same lateral position and different depth positions are associated with different resolutions, and wherein the particular resolution is determined as the resolution that is associated with at least one particular zone that includes both a lateral component and a depth component of the virtual object position; and causing the display to present the virtual object at the virtual object position and at the particular resolution.