# US 10,482,648 B2 · Scene-based foveated rendering of graphics content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,648 B2 - Document ID: US10482648B2 - Status: Granted - Kind: B2 - Application number: 15/377,488 - Filed: 2016-12-13 - Published: 2018-06-14 - Granted: 2019-11-19 - Earliest claimed: 2016-12-13 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Mehrad Tavakoli; Moinul Khan; Martin Renschler; Mriganka Mondal - Web page: https://patentgrep.com/patent/US10482648B2 ## CPC - G06T 15/00: Three-dimensional [3D] image rendering - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06T 3/04: Context-preserving transformations, e.g. by using an importance map (panospheric to cylindrical image transformations G06T3/12) - G06T 11/40: Filling planar surfaces by adding surface attributes, e.g. adding colours or textures - G06V 40/18: Eye characteristics, e.g. of the iris - G06T 2210/36: Level of detail - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An apparatus is configured to render graphics content to reduce latency of the graphics content. The apparatus includes a display configured to present graphics content including a first portion corresponding to an area of interest and further including a second portion. The apparatus further includes a fovea estimation engine configured to generate an indication of the area of interest based on scene information related to the graphics content. The apparatus further includes a rendering engine responsive to the fovea estimation engine. The rendering engine is configured to perform a comparison of a first result of an evaluation metric on part of the area of interest with a second result of the evaluation metric with another part of the area of interest. The rendering engine is further configured to render the graphics content using predictive adjustment to reduce latency based on the comparison. ## Claims ### Claim 1 An apparatus configured to render graphics content, the apparatus comprising: a fovea estimation engine configured to generate an indication of a first portion of graphics content based on scene information related to the graphics content, the graphics content further including a second portion; and a rendering engine responsive to the fovea estimation engine, the rendering engine configured to perform a comparison of a first value of an evaluation metric associated with the graphics content with a second value of the evaluation metric, the first value associated with the first portion and the second value associated with the second portion, wherein the evaluation metric is based on at least one of a number of visible triangles associated with a bin of the graphics content, a number of pixels updated per triangle of the graphics content, a number of texture samples read per triangle of the graphics content, or a number of arithmetic logic unit (ALU) instructions associated per triangle or bin of the graphics content, wherein the rendering engine is further configured to render the graphics content using predictive adjustment based on the comparison. ### Claim 2 (depends on 1) The apparatus of claim 1, further comprising a processor, the processor including the fovea estimation engine and the rendering engine. ### Claim 3 (depends on 1) The apparatus of claim 1, further comprising a camera configured to generate eye position information, wherein the fovea estimation engine is further configured to identify the first portion further based on the eye position information. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the rendering engine is further configured to render the graphics content by decreasing one or more of a fidelity of the second portion, an amount of detail of the second portion, or a resolution of the second portion. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the rendering engine is further configured to render the graphics content by increasing one or more of a fidelity of the first portion, an amount of detail of the first portion, or a resolution of the first portion. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the evaluation metric is further based on a ranking of a bin that is part of the graphics content. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the evaluation metric is further based on a number of drawcalls in a bin that is part of the graphics content. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the evaluation metric is further based on a number of pixels to be shaded of a bin. ### Claim 9 (depends on 1) The apparatus of claim 1, wherein the evaluation metric is further based on a number of updated pixels associated with a bin that is part of the graphics content. ### Claim 10 (depends on 1) The apparatus of claim 1, wherein the evaluation metric is further based on a number of drawcalls associated with visible primitives of a bin. ### Claim 11 (depends on 1) The apparatus of claim 1, wherein the scene information includes a set of values associated with a triangle of the graphics content. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the set of values indicates a number of updated pixels associated with the triangle, a number of read texture samples associated with triangle, a triangle size of the triangle, a number of arithmetic logic unit (ALU) instructions associated with the triangle, one or more other parameters associated with the triangle, or a combination thereof. ### Claim 13 (depends on 1) The apparatus of claim 1, wherein the scene information indicates a drawcall pattern associated with frames of the graphics content. ### Claim 14 A method of operation of a device, the method comprising: generating an indication of a first portion of graphics content at a device based on scene information related to the graphics content, the graphics content further including a second portion; and rendering the graphics content at the device based on comparing a first value of an evaluation metric associated with the graphics content with a second value of the evaluation metric, the first value associated with the first portion and the second value associated with the second portion, wherein the evaluation metric is based on at least one of a number of visible triangles associated with a bin of the graphics content, a number of pixels updated per triangle of the graphics content, a number of texture samples read per triangle of the graphics content, or a number of arithmetic logic unit (ALU) instructions associated per triangle or bin of the graphics content, wherein the graphics content is rendered using predictive adjustment based on the comparison. ### Claim 15 (depends on 14) The method of claim 14, further comprising receiving eye position information from a camera, wherein the first portion is identified further based on the eye position information. ### Claim 16 (depends on 14) The method of claim 14, wherein the scene information is received from a developer of the graphics content. ### Claim 17 (depends on 14) The method of claim 14, wherein the evaluation metric is further based on a number of visible drawcalls per bin of the graphics content, and further comprising identifying the first portion based on a number of visible drawcalls per bin of the first portion. ### Claim 18 (depends on 14) The method of claim 14, further comprising identifying the first portion based on a number of per bin of the first portion. ### Claim 19 (depends on 14) The method of claim 14, wherein the evaluation metric is further based on a number of pixels updated per bin of the graphics content, and further comprising identifying the first portion based on a number of updated pixels per bin of the first portion. ### Claim 20 (depends on 14) The method of claim 14, further comprising identifying the first portion based on a number of ALU instructions per bin of the first portion. ### Claim 21 (depends on 14) The method of claim 14, further comprising identifying the first portion based on a number of updated pixels per triangle of the first portion. ### Claim 22 (depends on 14) The method of claim 14, further comprising identifying the first portion based on a number of texture samples read per triangle of the first portion. ### Claim 23 (depends on 14) The method of claim 14, wherein the evaluation metric is further based on a triangle size associated with the graphics content, and further comprising identifying the first portion based on a triangle size of one or more triangles of the first portion. ### Claim 24 (depends on 14) The method of claim 14, further comprising identifying the first portion based on a number of ALU instructions per triangle of the first portion. ### Claim 25 (depends on 14) The method of claim 14, wherein the scene information is generated during a binning process, and wherein the first portion is foveated during a rendering process that occurs after the binning process. ### Claim 26 (depends on 14) The method of claim 14, further comprising receiving eye position information, wherein the first portion is identified further based on the eye position information. ### Claim 27 (depends on 14) The method of claim 14, wherein the graphics content is rendered using predictive adjustment to reduce latency associated with foveating the first portion, and further comprising, after rendering the graphics content, outputting the graphics content to be presented at a display device. ### Claim 28 A method of operation of a device, the method comprising: generating an indication of a first portion of graphics content at a device based on scene information related to the graphics content, the graphics content further including a second portion; identifying a first set of drawcalls of a first frame of the graphics content; identifying a second set of drawcalls of a second frame of the graphics content, the second set of drawcalls including at least one drawcall that is not included in the first set of drawcalls, wherein the first portion is identified in the second frame based on the at least one drawcall; and rendering the graphics content at the device based on comparing a first value of an evaluation metric associated with the graphics content with a second value of the evaluation metric, the first value associated with the first portion and the second value associated with the second portion, wherein the first value is determined based at least in part on a number of pixels associated with the first portion that are updated during rendering of the graphics content. ### Claim 29 An apparatus comprising: means for generating an indication of a first portion of graphics content based on scene information related to the graphics content, the graphics content further including a second portion; and means for rendering the graphics content based on comparing a first value of an evaluation metric associated with the graphics content with a second value of the evaluation metric, the first value associated with the first portion and the second value associated with the second portion, wherein the evaluation metric is based on at least one of a number of visible triangles associated with a bin of the graphics content, a number of pixels updated per triangle of the graphics content, a number of texture samples read per triangle of the graphics content, or a number of arithmetic logic unit (ALU) instructions associated per triangle or bin of the graphics content, and wherein the means for rendering is configured to render the graphics content using predictive adjustment based on the comparison. ### Claim 30 (depends on 29) The apparatus of claim 29, further comprising means for performing a binning process associated with processing of the graphics content and for determining the scene information during the binning process.