# US 11,308,658 B2 · Motion adaptive rendering using variable rate shading > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,658 B2 - Document ID: US11308658B2 - Status: Granted - Kind: B2 - Application number: 17/150,585 - Filed: 2021-01-15 - Published: 2021-06-03 - Granted: 2022-04-19 - Earliest claimed: 2018-08-09 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Lei Yang; Emmett Michael Kilgariff; Eric Brian Lum - Web page: https://patentgrep.com/patent/US11308658B2 ## CPC - G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) - G06T 5/73: Deblurring; Sharpening - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06T 7/90: Determination of colour characteristics - G06T 13/20: Three-dimensional [3D] animation - G06T 13/80: Two-dimensional [2D] animation, e.g. using sprites - G06T 15/06: Ray-tracing - G06T 15/80: Shading - G06T 2207/20201: Motion blur correction - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Motion adaptive shading increases rendering performance for real-time animation in graphics systems while maintaining dynamic image quality. Each frame of an animation is statically displayed within a refresh interval, while a viewer's eyes move continuously relative to the image when actively tracking a moving object being displayed. As a result, a statically displayed frame is essentially smeared across the viewer's continuously moving retina over the lifetime of the frame, causing a perception of blur referred to as an eye-tracking motion blur effect. A region of an image depicting a moving object may be rendered at a lower shading rate because eye-tracking motion blur will substantially mask any blur introduced by reducing the shading rate. Reducing an average shading rate for rendering frames reduces computational effort per frame and may advantageously allow a rendering system to operate at a higher frame rate to provide a smoother, clearer visual experience. ## Claims ### Claim 1 A computer-implemented method for motion adaptive rendering, comprising: calculating a horizontal motion value for a first region of a frame, wherein the frame includes multiple regions; calculating a vertical motion value for the first region; comparing the horizontal motion value to a first plurality of threshold values to determine a horizontal shading dimension, wherein the horizontal shading dimension has a first value when the horizontal motion value is less than all of the first plurality of threshold values, the horizontal shading dimension has a second value when the horizontal motion value is greater than all of the first plurality of threshold values, and the horizontal shading dimension has a third value when the horizontal motion value is less than some of the first plurality of threshold values and greater than some of the first plurality of threshold values; comparing the vertical motion value to a second plurality of threshold values to determine a vertical shading dimension, wherein the vertical shading dimension has a fourth value when the vertical motion value is less than all of the second plurality of threshold values, the vertical shading dimension has a fifth value when the vertical motion value is greater than all of the second plurality of threshold values, and the vertical shading dimension has a sixth value when the vertical motion value is less than some of the second plurality of threshold values and greater than some of the second plurality of threshold values; subdividing the first region into a plurality of pixel blocks, wherein each pixel block has a number of pixels along the horizontal dimension of the pixel block that corresponds to the horizontal shading dimension and each pixel block has a number of pixels along the vertical dimension of the pixel block that corresponds to the vertical shading dimension; and rendering a graphics primitive by assigning a shading value to each pixel in at least one of the pixel blocks that is at least partially covered by the graphics primitive. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the first plurality of threshold values is the same as the second plurality of threshold values. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein the first value is the same as the fourth value, the second value is the same as the fifth value, and the third value is the same as the sixth value. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the horizontal motion value and the vertical motion value are calculated according to optical flow of color image data between two frames rendered immediately prior to the frame. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the horizontal motion value and the vertical motion value are calculated according to a camera position change or object position change between an immediately prior frame and the frame. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising: computing content variation for the first region based on an immediate previously rendered frame; and modifying the first plurality of threshold values and the second plurality of threshold values based on the content variation. ### Claim 7 (depends on 6) The computer-implemented of claim 6, wherein modifying the first plurality of threshold values and the second plurality of threshold values increases at least one of the horizontal shading dimension and the vertical shading dimension when the content variation indicates a low variation of color and luminance. ### Claim 8 (depends on 6) The computer-implemented of claim 6, wherein modifying the first plurality of threshold values and the second plurality of threshold values decreases at least one of the horizontal shading dimension and the vertical shading dimension when the content variation indicates a high variation of color and luminance. ### Claim 9 A system, comprising: a processor configured to: calculate a horizontal motion value for a first region of a frame, wherein the frame includes multiple regions; calculate a vertical motion value for the first region; compare the horizontal motion value to a first plurality of threshold values to determine a horizontal shading dimension, wherein the horizontal shading dimension has a first value when the horizontal motion value is less than all of the first plurality of threshold values, the horizontal shading dimension has a second value when the horizontal motion value is greater than all of the first plurality of threshold values, and the horizontal shading dimension has a third value when the horizontal motion value is less than some of the first plurality of threshold values and greater than some of the first plurality of threshold values; compare the vertical motion value to a second plurality of threshold values to determine a vertical shading dimension, wherein the vertical shading dimension has a fourth value when the vertical motion value is less than all of the second plurality of threshold values, the vertical shading dimension has a fifth value when the vertical motion value is greater than all of the second plurality of threshold values, and the vertical shading dimension has a sixth value when the vertical motion value is less than some of the second plurality of threshold values and greater than some of the second plurality of threshold values; subdivide the first region into a plurality of pixel blocks, wherein each pixel block has a number of pixels along the horizontal dimension of the pixel block that corresponds to the horizontal shading dimension and each pixel block has a number of pixels along the vertical dimension of the pixel block that corresponds to the vertical shading dimension; and render a graphics primitive by assigning a shading value to each pixel in at least one of the pixel blocks that is at least partially covered by the graphics primitive. ### Claim 10 (depends on 9) The system of claim 9, wherein the first plurality of threshold values is the same as the second plurality of threshold values. ### Claim 11 (depends on 10) The system of claim 10, wherein the first value is the same as the fourth value, the second value is the same as the fifth value, and the third value is the same as the sixth value. ### Claim 12 (depends on 9) The system of claim 9, wherein the horizontal motion value and the vertical motion value are calculated according to optical flow of color image data between two frames rendered immediately prior to the frame. ### Claim 13 (depends on 9) The system of claim 9, wherein the horizontal motion value and the vertical motion value are calculated according to a camera position change or object position change between an immediately prior frame and the frame. ### Claim 14 (depends on 9) The system of claim 9, further the processor is further configured to: compute content variation for the first region based on an immediate previously rendered frame; and modify the first plurality of threshold values and the second plurality of threshold values based on the content variation. ### Claim 15 (depends on 14) The system of claim 14, wherein modifying the first plurality of threshold values and the second plurality of threshold values increases at least one of the horizontal shading dimension and the vertical shading dimension when the content variation indicates a low variation of color and luminance. ### Claim 16 (depends on 14) The system of claim 14, wherein modifying the first plurality of threshold values and the second plurality of threshold values decreases at least one of the horizontal shading dimension and the vertical shading dimension when the content variation indicates a high variation of color and luminance. ### Claim 17 A non-transitory computer-readable media storing computer instructions for motion adaptive rendering that, when executed by one or more processors, cause the one or more processors to perform the steps of: calculating a horizontal motion value for a first region of a frame, wherein the frame includes multiple regions; calculating a vertical motion value for the first region; comparing the horizontal motion value to a first plurality of threshold values to determine a horizontal shading dimension, wherein the horizontal shading dimension has a first value when the horizontal motion value is less than all of the first plurality of threshold values, the horizontal shading dimension has a second value when the horizontal motion value is greater than all of the first plurality of threshold values, and the horizontal shading dimension has a third value when the horizontal motion value is less than some of the first plurality of threshold values and greater than some of the first plurality of threshold values; comparing the vertical motion value to a second plurality of threshold values to determine a vertical shading dimension, wherein the vertical shading dimension has a fourth value when the vertical motion value is less than all of the second plurality of threshold values, the vertical shading dimension has a fifth value when the vertical motion value is greater than all of the second plurality of threshold values, and the vertical shading dimension has a sixth value when the vertical motion value is less than some of the second plurality of threshold values and greater than some of the second plurality of threshold values; subdividing the first region into a plurality of pixel blocks, wherein each pixel block has a number of pixels along the horizontal dimension of the pixel block that corresponds to the horizontal shading dimension and each pixel block has a number of pixels along the vertical dimension of the pixel block that corresponds to the vertical shading dimension; and rendering a graphics primitive by assigning a shading value to each pixel in at least one of the pixel blocks that is at least partially covered by the graphics primitive. ### Claim 18 (depends on 17) The non-transitory computer-readable media of claim 17, wherein the first plurality of threshold values is the same as the second plurality of threshold values. ### Claim 19 (depends on 18) The non-transitory computer-readable media of claim 18, wherein the first value is the same as the fourth value, the second value is the same as the fifth value, and the third value is the same as the sixth value. ### Claim 20 (depends on 17) The non-transitory computer-readable media of claim 17, wherein the horizontal motion value and the vertical motion value are calculated according to optical flow of color image data between two frames rendered immediately prior to the frame.