# US 12,763,041 B2 · Low latency synchronization of video rendering pipelines with high refresh rates > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,763,041 B2 - Document ID: US12763041B2 - Status: Granted - Kind: B2 - Application number: 18/373,748 - Filed: 2023-09-27 - Published: 2025-03-27 - Granted: 2026-10-13 - Earliest claimed: 2023-09-27 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Viktor Grigoryevich Vandanov; Rouslan Lyubomirov Dimitrov; Seth Schneider - Web page: https://patentgrep.com/patent/US12763041B2 ## CPC - G09G 5/395: Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen (G09G5/399 takes precedence) - G09G 2340/0435: Change or adaptation of the frame rate of the video stream - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract Disclosed are apparatuses, systems, and techniques that reduce latency of frame processing pipelines. The techniques include but are not limited to causing a display to set a refresh rate that matches a frame rendering rate of an application and rendering, with the frame rendering rate, a plurality of frames. Rendering frames includes generating, using a first processing unit, sets of instructions associated with respective frames and generated starting at times spaced with the refresh rate, processing, using a second processing unit, the sets of instructions to render the frames, and causing the display to display the rendered frames. ## Claims ### Claim 1 A method comprising: rendering a plurality of frames at a frame rendering rate corresponding to a video application, the frame rendering rate matching a refresh rate of a display device, wherein rendering the plurality of frames comprises: generating, using a first processing unit, a plurality of sets of instructions for a second processing unit, wherein individual sets of instructions of the plurality of sets of instructions are associated with rendering, using the second processing unit, of respective frames of the plurality of frames and are generated at times spaced with the refresh rate, wherein the first processing unit generates the plurality of sets of instructions at a rate determined to maintain a queue of unexecuted, by the second processing unit, instructions that comprises one or more sets of instructions for rendering two or fewer frames of the plurality of frames; processing, using the second processing unit, the plurality of sets of instructions to render the plurality of frames; and causing the display device to display the plurality of frames. ### Claim 2 (depends on 1) The method of claim 1, wherein the display device comprises a variable refresh rate display. ### Claim 3 (depends on 1) The method of claim 1, wherein the frame rendering rate of the video application is set by: rendering an initial plurality of frames at an initial frame rendering rate to cause the display device to (i) identify the initial frame rendering rate and (ii) set, responsive to the identified initial frame rendering rate, the refresh rate; determining a difference between the refresh rate used by the display device and the initial frame rendering rate; and causing the video application to transition from the initial frame rendering rate to the frame rendering rate. ### Claim 4 (depends on 1) The method of claim 1, wherein the first processing unit comprises a central processing unit (CPU), and wherein the second processing unit comprises a graphics processing unit (GPU). ### Claim 5 (depends on 1) The method of claim 1, wherein the refresh rate is between 239 Hz and 241 Hz, between 164 Hz and 166 Hz, or between 359 Hz and 361 Hz. ### Claim 6 (depends on 1) The method of claim 1, wherein causing the display device to display the plurality of frames comprises: encoding, based at least on a video codec, at least one frame of the plurality of frames. ### Claim 7 (depends on 6) The method of claim 6, wherein causing the display device to display the plurality of frames further comprises: packetizing at least one encoded frame of the plurality of frames; and causing at least one packetized frame of the plurality of frames to be communicated to the display device. ### Claim 8 (depends on 1) The method of claim 1, wherein causing the display device to display the plurality of frames comprises: generating a plurality of packets wherein one or more individual packets of the plurality of packets comprise a representation of a single frame of the plurality of frames. ### Claim 9 (depends on 1) The method of claim 1, wherein the video application comprises at least one of a gaming application, a content streaming application, or a collaborative content creation application. ### Claim 10 A method comprising: determining a frame rendering rate associated with an initial plurality of frames rendered by a video application; setting a refresh rate of a display device to match the frame rendering rate; rendering an initial plurality of frames on the display device; communicating, to the video application, timing metrics associated with the rendered initial plurality of frames; receiving a plurality of frames rendered by the video application with an adjusted, based on the timing metrics, rendering rate; responsive to receiving a first frame of the plurality of frames prior to a first scheduled time for displaying the first frame, presenting the first frame on the display device concurrently with displaying a preceding frame of the plurality of frames; and responsive to receiving a second frame of the plurality of frames with a predetermined delay after a second scheduled time for displaying the second frame, discarding the second frame, wherein the predetermined delay is between 1.2 and 1.7 of a spacing between frames that corresponds to an inverse of the refresh rate. ### Claim 11 (depends on 10) The method of claim 10, wherein the display device comprises a variable refresh rate display. ### Claim 12 (depends on 10) The method of claim 10, wherein the refresh rate of the display device is set to be within 1 Hz from the determined frame rendering rate associated with the initial plurality of frames. ### Claim 13 (depends on 10) The method of claim 10, wherein the refresh rate is between 239 Hz and 241 Hz, between 164 Hz and 166 Hz, or between 359 Hz and 361 Hz. ### Claim 14 (depends on 10) The method of claim 10, wherein receiving the plurality of frames comprises: receiving a plurality of packets; depacketizing the plurality of packets to obtain a plurality of encoded frames; and decoding the plurality of encoded frames to recover the plurality of frames. ### Claim 15 (depends on 14) The method of claim 14, wherein one or more individual packets of the plurality of packets comprise an encoded representation of a single frame of the plurality of frames. ### Claim 16 A system comprising: a plurality of processing units to: render, a plurality of frames at a frame rendering rate of an application, the frame rendering rate matching a refresh rate of a display device, wherein to render the plurality of frames, the plurality of processing units are to: generate, using a first processing unit of the plurality of processing units, a plurality of sets of instructions for a second processing unit of the plurality of processing units, wherein individual sets of instructions of the plurality of sets of instructions are associated with rendering, using the second processing unit, of respective frames of the plurality of frames and are generated starting at times spaced with the refresh rate, wherein the first processing unit is to generate the plurality of sets of instructions at a rate determined to maintain a queue of unexecuted, by the second processing unit, instructions that comprises one or more sets of instructions for rendering two or fewer frames of the plurality of frames; process, using the second processing unit, the plurality of sets of instructions to render the plurality of frames; and cause the display device to display the plurality of frames. ### Claim 17 (depends on 16) The system of claim 16, wherein the display device comprises a variable refresh rate display. ### Claim 18 (depends on 16) The system of claim 16, wherein the frame rendering rate of the application is set by: rendering an initial plurality of frames at an initial frame rendering rate to cause the display device to (i) identify the initial frame rendering rate and (ii), responsive to the identified initial frame rendering rate, the refresh rate; determining a difference between the refresh rate used by the display device and the initial frame rendering rate; and causing the application to transition from the initial frame rendering rate to the frame rendering rate. ### Claim 19 (depends on 16) The system of claim 16, wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system for generating or presenting at least one of augmented reality content, virtual reality content, or mixed reality content; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data using AI operations; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.