# US 10,319,065 B2 · Intra-frame real-time frequency control

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,319,065 B2
- Document ID: US10319065B2
- Status: Granted
- Kind: B2
- Application number: 15/486,809
- Filed: 2017-04-13
- Published: 2018-10-18
- Granted: 2019-06-11
- Earliest claimed: 2017-04-13
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Hee jun Park; Ted E. Williams
- Web page: https://patentgrep.com/patent/US10319065B2

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 1/324: by lowering clock frequency
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G09G 5/001: {Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor}
- G09G 5/363: {Graphics controllers}
- G09G 5/395: Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen (G09G5/399 takes precedence)
- G09G 2330/021: Power management, e.g. power saving
- G09G 2340/0435: Change or adaptation of the frame rate of the video stream
- G09G 2350/00: Solving problems of bandwidth in display systems
- G09G 2360/08: Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

A recommend operating frequency for a GPU is determine by an operating system. Timing information is used to estimate a time to complete the rendering of the current frame. If the estimate of the time to complete the rendering of the current frame is greater than the time left before the deadline, the operating frequency of the GPU is rapidly increased mid-frame. The new operating frequency is selected so that the rendering of the current frame can be expected to complete before the deadline arrives. Thus, when the rendering workload is low for a frame, the GPU is operated at a lower frequency thereby saving power. When the workload is high for a frame (even if the workload was lower for the previous frame), the GPU is operated at a higher frequency thereby ensuring a quality user experience.

## Claims

### Claim 1

An integrated circuit, comprising:
a graphics processing unit (GPU) to periodically render frame images in a frame buffer intended for output to a display device; and,
a frequency management unit to periodically control, on a frame by frame basis, a processing frequency of the GPU based on a frame by frame estimate of a processing time for a current frame and an indicator of a remaining time before a target time to complete rendering of the current frame arrives, wherein the GPU processing frequency is controlled based on the estimated time remaining to complete the rendering of the current frame exceeding the time remaining before the deadline for completing the rendering of the current frame.

### Claim 2 (depends on 1)

The integrated circuit of claim 1, wherein the frequency management unit is to set a minimum processing frequency based on an indicator controlled by an operating system component.

### Claim 3 (depends on 1)

The integrated circuit of claim 1, wherein the target time to complete rendering of the current frame is based on an indicator of a remaining time until a vertical synchronization signal occurs.

### Claim 4 (depends on 1)

The integrated circuit of claim 1, wherein the estimate of the processing time for the current frame is based on measured indicators of a plurality of processing times associated with corresponding plurality of previously rendered frames.

### Claim 5 (depends on 1)

The integrated circuit of claim 1, wherein the estimate of the processing time for the current frame is based on an average of a plurality of processing times associated with recently rendered frames.

### Claim 6 (depends on 1)

The integrated circuit of claim 1, wherein the frequency management unit is to change the processing frequency of the GPU after the GPU has started rendering the current frame.

### Claim 7 (depends on 1)

The integrated circuit of claim 1, wherein the frequency management unit is to change the processing frequency of the GPU before the GPU has started rendering the current frame.

### Claim 8

A method of operating a graphics processing unit (GPU) to periodically, at a frame rate, render frame images in a frame buffer intended for output to a display device, comprising:
operating the GPU at a first processing frequency;
for each frame and occurring at least at the frame rate, while a current frame is being rendered, estimating a time remaining to complete a rendering of the current frame;
for each frame and occurring at least at the frame rate, receiving an indicator of a time remaining before a deadline for completing the rendering of the current frame; and,
while the current frame is being rendered, and based on the estimated time remaining to complete the rendering of the current frame, operating the GPU at a second operating frequency for at least a portion of the time remaining before the deadline for completing the rendering of the current frame arrives, wherein the GPU is operated at the second operating frequency based on one of: 1) the estimated time remaining to complete the rendering of the current frame exceeding the time remaining before the deadline for completing the rendering of the current frame, and 2) the time remaining before the deadline for completing the rendering of the current frame exceeding the estimated time remaining to complete the rendering of the current frame while operating at the first operating frequency.

### Claim 9 (depends on 8)

The method of claim 8, wherein the first processing frequency is determined by an operating system.

### Claim 10 (depends on 8)

The method of claim 8, wherein the deadline for completing the rendering of the current frame is based on a vertical synchronization signal.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
while a successive frame is being rendered, estimating a second time remaining to complete a rendering of the successive frame;
receiving a second indicator of a second time remaining before a second deadline for completing the rendering of the successive frame; and,
while the successive frame is being rendered, and based on the second estimated time remaining to complete the rendering of the successive frame, operating the GPU at the first operating frequency for at least a second portion of the second time remaining before the deadline for completing the rendering of the successive frame arrives.

### Claim 12 (depends on 8)

The method of claim 8, wherein the time remaining to complete the rendering of the current frame is based on a plurality of rendering times taken to render immediately preceding frames.

### Claim 13

A method of operating a graphics processing unit (GPU) to periodically, at a frame rate, render frame images in a frame buffer intended for output to a display device, comprising, for each frame:
before receiving a command to begin rendering a current frame, operating the GPU at a first processing frequency;
before receiving the command to begin rendering the current frame, estimating a first time remaining to completely rendering the current frame;
receiving an indicator of the first time remaining before a first deadline for completing the rendering of the current frame; and,
before receiving the command to begin rendering the current frame, and based on the estimated time remaining to completely render the current frame, operating the GPU at a second operating frequency for at least a portion of the time remaining before the arrival of the first deadline for completely rendering the current frame.

### Claim 14 (depends on 13)

The method of claim 13, wherein the GPU is operated at the second operating frequency based on the estimated time remaining to completely render the current frame exceeding the time remaining before the first deadline for completing the rendering of the current frame.

### Claim 15 (depends on 13)

The method of claim 13, wherein the GPU is operated at the second operating frequency based on the time remaining before the first deadline for completing the rendering of the current frame exceeding the estimated time remaining to completely render the current frame.

### Claim 16 (depends on 13)

The method of claim 13, wherein the first processing frequency is determined by an operating system.

### Claim 17 (depends on 13)

The method of claim 13, wherein the time remaining to completely render the current frame is based on a plurality of rendering times taken to render immediately preceding frames.

### Claim 18 (depends on 13)

The method of claim 13, wherein the command is a first command, the method further comprising:
after receiving the first command to begin rendering the current frame and before receiving a second command to begin rendering a successive frame, operating the GPU at the second processing frequency;
before receiving the second command to begin rendering the successive frame, estimating a second time remaining to complete a rendering of the successive frame;
receiving an indicator of a second time remaining before a second deadline for completing the rendering of the successive frame; and,
before receiving the second command to begin rendering the successive frame, and based on the estimated time remaining to complete the rendering of the successive frame, operating the GPU at the first operating frequency for at least a portion of the time remaining before the second deadline for completing the rendering of the successive frame arrives.