# US 9,436,970 B2 · Display co-processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,436,970 B2
- Document ID: US9436970B2
- Status: Granted
- Kind: B2
- Application number: 14/200,213
- Filed: 2014-03-07
- Published: 2014-09-18
- Granted: 2016-09-06
- Earliest claimed: 2013-03-15
- Assignees: Google Technology Holdings LLC
- Applicants: Google Technology Holdings LLC
- Inventors: Nathan M. Connell; Christian L. Flowers; John W. Kaehler; George B. Standish
- Web page: https://patentgrep.com/patent/US9436970B2

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 1/3293: by switching to a less power-consuming processor, e.g. sub-CPU
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G06F 9/5044: {considering hardware capabilities}
- G09G 5/006: {Details of the interface to the display terminal (specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04)}
- G06F 3/1415: {with means for detecting differences between the image stored in the host and the images displayed on the displays}
- G09G 5/005: {Adapting incoming signals to the display format of the display terminal}
- G09G 5/363: {Graphics controllers}
- G09G 2310/04: Partial updating of the display screen
- G09G 2320/0673: for control of gamma adjustment, e.g. selecting another gamma curve
- G09G 2330/021: Power management, e.g. power saving
- G09G 2340/12: Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
- G09G 2360/06: Use of more than one graphics processor to process data before displaying to one or more screens
- G09G 2360/18: Use of a frame buffer in a display terminal, inclusive of the display panel
- G09G 2370/10: Use of a protocol of communication by packets in interfaces along the display data pipeline
- G09G 2370/20: Details of the management of multiple sources of image data
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- Y02D 30/50: in wire-line communication networks, e.g. low power modes or reduced link rate
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

In embodiments of display co-processing, a computing device includes a display, a full-power processor, and a low-power processor that can alter visual content presented by the display without utilizing the full-power processor. The low-power processor can, responsive to a request from the full-power processor, generate additional display data to update display data stored in a frame-buffer of the display. The low-power processor can then transmit the additional display data to the frame-buffer effective to alter at least a portion of the visual content presented by the display. In some embodiments, the additional display data is transmitted via a protocol converter that forwards the display data to the display using a display-specific communication protocol.

## Claims

### Claim 1

A method comprising:
rendering and transmitting, by an application processor, display data to a display;
causing, by a display manager based on a computing device entering a state of low graphical activity, the application processor to write data to control registers of the low-power processor and enter a low-power state;
decreasing, by the display based on the computing device entering the state of low graphical activity, a refresh-frequency of visual content; and
generating and sending to the display, by a low-power processor based on the computing device entering the state of low graphical activity and based on the data stored in the control registers by the application processor, additional display data to alter at least a portion of visual content presented by the display.

### Claim 2 (depends on 1)

The method as recited in claim 1, wherein the application processor includes a graphics processing unit (GPU).

### Claim 3

A method comprising:
receiving, by a low-power processor from an application processor, a request to alter visual content presented by a display;
generating, based on the request, additional display data to alter the visual content presented by the display;
causing a protocol converter to initiate a data transfer session with the display, the protocol converter configured to forward data to the display using a display-specific communication protocol, the data transfer session including signaling a mobile industry processor interface (MIPI) compliant data transfer initiation sequence or a MIPI compliant data transfer termination sequence;
transmitting, during the data transfer session, the additional display data to the protocol converter; and
causing the protocol converter to terminate the data transfer session effective to cause the display to update the visual content based on the additional display data.

### Claim 4 (depends on 3)

The method as recited in claim 3, wherein the MIPI compliant data transfer initiation sequence or MIPI compliant data transfer termination sequence comprises the MIPI compliant data transfer initiation sequence.

### Claim 5 (depends on 3)

The method as recited in claim 3, wherein the MIPI compliant data transfer initiation sequence or MIPI compliant data transfer termination sequence comprises the MIPI compliant data transfer termination sequence.

### Claim 6 (depends on 3)

The method as recited in claim 3, further comprising altering the visual content presented by the display by altering a luminance or gamma of at least a portion of the visual content.

### Claim 7 (depends on 3)

The method as recited in claim 3, further comprising, prior to generating the additional display data, accessing additional information describing luminance and timing parameters for altering the visual content presented by the display.

### Claim 8 (depends on 7)

The method as recited in claim 7, wherein the additional information is stored in data registers accessible by the low-power processor.

### Claim 9 (depends on 3)

The method as recited in claim 3, further comprising transmitting, to the application processor, an indication of a status of the visual content presented by the display.

### Claim 10

A system comprising:
a display configured to present visual content based on display data stored in a frame-buffer;
a first processor configured to, upon entry of the system into a low-power state, write data describing alteration of the visual content to control registers; and
a second processor configured to:
generate, based on the data written to the control registers, additional display data to update less than all of the display data stored in the frame-buffer; and
write the additional display data to the frame-buffer of the display to alter at least a portion of the visual content presented by the display.

### Claim 11 (depends on 10)

The system as recited in claim 10, wherein the first processor and the second processor write the respective display data and additional display data to the frame-buffer via a de-multiplexor operably coupled to a display-specific interface of the display.

### Claim 12 (depends on 10)

The system as recited in claim 10, wherein:
the second processor does not include a display-specific data interface,
the system further includes a protocol converter configured to forward the display data to the display using a display-specific communication protocol, and
the additional display data is communicated to the protocol converter that is configured to write the additional display data to the frame-buffer.

### Claim 13 (depends on 12)

The system as recited in claim 12, wherein the additional display data alters a gamma or luminescence of the visual content.

### Claim 14 (depends on 12)

The system as recited in claim 12, wherein the second processor is further configured to update display status registers to track alterations of the visual content.

### Claim 15 (depends on 14)

The system as recited in claim 14, wherein the first processor is further configured to access the display status registers and determine whether to request further alterations to the visual content by the second processor based on the display status registers.

### Claim 16 (depends on 1)

The method as recited in claim 1, wherein the data written to the control registers of the low-power processor by the application processor includes timing parameters for the low-power processor to alter the visual content.

### Claim 17 (depends on 1)

The method as recited in claim 1, wherein the data written to the control registers of the low-power processor by the application processor includes an off time specifying an amount of time between alterations for the low-power processor to alter the visual content.

### Claim 18 (depends on 1)

The method as recited in claim 1, wherein the data written to the control registers of the low-power processor by the application processor includes a partial region register index indicating a start position within a region of the display.

### Claim 19 (depends on 1)

The method as recited in claim 1, further comprising the application processor pushing notifications to the low-power processor while the application processor is in the low-power state.

### Claim 20 (depends on 1)

The method as recited in claim 1, further comprising:
writing, by the low-power processor to status registers of the low-power processor, display status data tracking a current state of alterations of the visual content; and
determining, by the application processor based on the status registers, whether the low-power processor successfully altered the visual content.

### Claim 21

A system comprising:
a low-power processor configured to:
receive, from an application processor, a request to alter visual content presented by a display;
generate, based on the request, additional display data to alter the visual content presented by the display; and
transmit, during a data transfer session, the additional display data to a protocol converter; and
the protocol converter configured to:
initiate a data transfer session with the display, the data transfer session including signaling a mobile industry processor interface (MIPI) compliant data transfer initiation sequence or a MIPI compliant data transfer termination sequence;
forward the additional display data to the display using a display-specific communication protocol; and
terminate the data transfer session effective to cause the display to update the visual content based on the additional display data.