# US 2024/0203376 A1 · Efficiently processing multiple non-overlapping layer images in display processing units

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203376 A1
- Document ID: US20240203376A1
- Status: Published application
- Kind: A1
- Application number: 18/066,034
- Filed: 2022-12-14
- Published: 2024-06-20
- Earliest claimed: 2022-12-14
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Sreekanth Modaikkal; Kumar Saurabh; Gary Arthur Ciambella; Chun Wang; Anitha Madugiri Siddaraju
- Web page: https://patentgrep.com/patent/US20240203376A1

## CPC

- G09G 5/14: Display of multiple viewports
- G06T 1/60: Memory management
- G09G 5/026: {Control of mixing and/or overlay of colours in general (G09G5/022 and G09G5/024 take precedence)}
- G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence)
- G09G 5/397: Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay (G09G5/02 takes precedence)
- G09G 2340/10: Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
- G09G 2340/12: Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
- G09G 2360/12: Frame memory handling
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Efficiently processing multiple non-overlapping layer images in display processing units is disclosed herein. In this regard, in some exemplary aspects, a display processing unit comprising a plurality of memory access pipeline circuits and a layer mixer circuit is provided. For each non-overlapping layer image of a plurality of non-overlapping layer images, a memory access pipeline circuit obtains image configuration data for the non-overlapping layer image, and fetches the non-overlapping layer image from an image data storage device based on the image configuration data. The memory access pipeline circuit then outputs each pixel of the non-overlapping layer image as part of an intermediate preblend image data stream based on the image configuration data. The layer mixer circuit blends the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream, and outputs the display data stream to a display device.

## Claims

### Claim 1

A display processing unit, comprising:
a plurality of memory access pipeline circuits;
a layer mixer circuit;
a first memory access pipeline circuit of the plurality of memory access pipeline circuits configured to, for each non-overlapping layer image of a plurality of non-overlapping layer images:
obtain image configuration data for the non-overlapping layer image;
fetch the non-overlapping layer image from an image data storage device based on the image configuration data; and
output each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data; and
the layer mixer circuit configured to:
blend the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream; and
output the display data stream to a display device.

### Claim 2 (depends on 1)

The display processing unit of claim 1, wherein the layer mixer circuit is further configured to:
receive the intermediate preblend image data stream from the first memory access pipeline circuit; and
receive the background layer image data stream from a second memory access pipeline circuit of the plurality of memory access pipeline circuits.

### Claim 3 (depends on 1)

The display processing unit of claim 1, wherein the image configuration data for each non-overlapping layer image of the plurality of the non-overlapping layer images comprises one or more of a source image size, an output image size, a set of source image position coordinates, a set of output image position coordinates, a source address, a stride value, an image format, and an unpack pattern value.

### Claim 4 (depends on 1)

The display processing unit of claim 1, wherein:
the first memory access pipeline circuit further comprises an image configuration queue;
the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, receive, from a plurality of image configuration registers, the image configuration data into the image configuration queue; and
the first memory access pipeline circuit is configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtain the image configuration data from a top entry of the image configuration queue.

### Claim 5 (depends on 4)

The display processing unit of claim 4, wherein:
the first memory access pipeline circuit is further configured to:
calculate a set of position coordinates and a size of the intermediate preblend image, based on the image configuration data for the plurality of non-overlapping layer images; and
transmit the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit; and
the layer mixer circuit configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream based on the set of position coordinates and the size of the intermediate preblend image.

### Claim 6 (depends on 4)

The display processing unit of claim 4, wherein:
the first memory access pipeline circuit comprises a first selectable image configuration queue and a second selectable image configuration queue;
the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, receive, from a plurality of image configuration registers, the image configuration data into the first selectable image configuration queue;
the first memory access pipeline circuit is configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtain the image configuration data from a top entry of the first selectable image configuration queue; and
the first memory access pipeline circuit is further configured to, in parallel with obtaining the image configuration data from the top entry of the first selectable image configuration queue, for each non-overlapping layer image of a next plurality of non-overlapping layer images, receive, from the plurality of image configuration registers, next image configuration data into the second selectable image configuration queue.

### Claim 7 (depends on 1)

The display processing unit of claim 1, wherein the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to, for each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer image of the plurality of non-overlapping layer images, output, to the display data stream, a blend of the pixel with a corresponding pixel of the background layer image.

### Claim 8 (depends on 7)

The display processing unit of claim 7, wherein:
the first memory access pipeline circuit is further configured to, for each non-overlapping layer image of the plurality of non-overlapping layer images, output, to the intermediate preblend image data stream, a blend bypass pixel for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image; and
the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to, for each blend bypass pixel of the intermediate preblend image data stream, output, to the display data stream, a corresponding pixel of the background layer image.

### Claim 9 (depends on 8)

The display processing unit of claim 8, wherein the blend bypass pixel comprises a pixel having a pre-specified color value.

### Claim 10 (depends on 2)

The display processing unit of claim 2, wherein:
the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream by being configured to:
blend the intermediate preblend image data stream and the background layer image data stream as the display data stream within a blending stage of a plurality of blending stages, wherein each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits; and
output the display data stream of each blending stage preceding a final blending stage of the plurality of blending stages as the background layer image data stream to a successive blending stage; and
the layer mixer circuit is configured to output the display data stream to the display device by being configured to output the display data stream of the final blending stage of the plurality of blending stages to the display device.

### Claim 11 (depends on 10)

The display processing unit of claim 10, wherein:
the layer mixer circuit further comprises a plurality of layer mixer configuration queues;
the layer mixer circuit is further configured to:
receive a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit into a layer mixer configuration queue of the plurality of layer mixer configuration queues;
assign a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier; and
responsive to receiving the intermediate preblend image data stream from the first memory access pipeline circuit, obtain the memory access pipeline identifier and the blending stage indication from a top entry of the layer mixer configuration queue based on the tag; and
the layer mixer circuit is configured to blend the intermediate preblend image data stream and the background layer image data stream as the display data stream within the blending stage of the plurality of blending stages based on the blending stage indication.

### Claim 12 (depends on 1)

The display processing unit of claim 1, integrated into a device selected from the group consisting of: a set top box; an entertainment unit; a navigation device; a communications device; a fixed location data unit; a mobile location data unit; a global positioning system (GPS) device; a mobile phone; a cellular phone; a smart phone; a session initiation protocol (SIP) phone; a tablet; a phablet; a server; a computer; a portable computer; a mobile computing device; a wearable computing device; a desktop computer; a personal digital assistant (PDA); a monitor; a computer monitor; a television; a tuner; a radio; a satellite radio; a music player; a digital music player; a portable music player; a digital video player; a video player; a digital video disc (DVD) player; a portable digital video player; an automobile; a vehicle component; avionics systems; a drone; and a multicopter.

### Claim 13

A display processing unit, comprising:
means for, for each non-overlapping layer image of a plurality of non-overlapping layer images:
obtaining image configuration data for the non-overlapping layer image;
fetching the non-overlapping layer image from an image data storage device based on the image configuration data; and
outputting each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data;
means for blending the intermediate preblend image data stream and a background layer image data stream comprising a background layer image as a display data stream; and
means for outputting the display data stream to a display device.

### Claim 14

A method for efficiently processing multiple non-overlapping layer images, the method comprising:
for each non-overlapping layer image of a plurality of non-overlapping layer images:
obtaining, by a first memory access pipeline circuit of a plurality of memory access pipeline circuits of a display processing unit, image configuration data for the non-overlapping layer image;
fetching, by the first memory access pipeline circuit, the non-overlapping layer image from an image data storage device based on the image configuration data; and
outputting, by the first memory access pipeline circuit, each pixel of the non-overlapping layer image as part of an intermediate preblend image to an intermediate preblend image data stream based on the image configuration data;
blending, by a layer mixer circuit of the display processing unit, the intermediate preblend image data stream and a background layer image data stream as a display data stream; and
outputting, by the layer mixer circuit, the display data stream to a display device.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
receiving, by the layer mixer circuit, the intermediate preblend image data stream from the first memory access pipeline circuit; and
receiving, by the layer mixer circuit, the background layer image data stream comprising a background layer image from a second memory access pipeline circuit of the plurality of memory access pipeline circuits.

### Claim 16 (depends on 14)

The method of claim 14, wherein the image configuration data for each non-overlapping layer image of the plurality of the non-overlapping layer images comprises one or more of a source image size, an output image size, a set of source image position coordinates, a set of output image position coordinates, a source address, a stride value, an image format, and an unpack pattern value.

### Claim 17 (depends on 14)

The method of claim 14, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from a plurality of image configuration registers, the image configuration data into an image configuration queue of the first memory access pipeline circuit;
wherein, for each non-overlapping layer image of the plurality of non-overlapping layer images, obtaining the image configuration data for the non-overlapping layer image comprises obtaining the image configuration data from a top entry of the image configuration queue.

### Claim 18 (depends on 17)

The method of claim 17, further comprising:
calculating, by the first memory access pipeline circuit, a set of position coordinates and a size of the intermediate preblend image, based on the image configuration data for the plurality of non-overlapping layer images; and
transmitting, by the first memory access pipeline circuit, the set of position coordinates and the size of the intermediate preblend image to the layer mixer circuit;
wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream is based on the set of position coordinates and the size of the intermediate preblend image.

### Claim 19 (depends on 17)

The method of claim 17, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from a plurality of image configuration registers, the image configuration data into a first selectable image configuration queue;
wherein:
for each non-overlapping layer image of the plurality of non-overlapping layer images, obtaining the image configuration data comprises obtaining the image configuration data from a top entry of the first selectable image configuration queue; and
the method further comprises, in parallel with obtaining the image configuration data from the top entry of the first selectable image configuration queue, for each non-overlapping layer image of a next plurality of non-overlapping layer images, receiving, by the first memory access pipeline circuit from the plurality of image configuration registers, next image configuration data into a second selectable image configuration queue.

### Claim 20 (depends on 15)

The method of claim 15, wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises, for each pixel of the intermediate preblend image data stream that corresponds to a non-overlapping layer image of the plurality of non-overlapping layer images, outputting, by the layer mixer circuit to the display data stream, a blend of the pixel with a corresponding pixel of the background layer image.

### Claim 21 (depends on 15)

The method of claim 15, further comprising, for each non-overlapping layer image of the plurality of non-overlapping layer images, outputting, by the first memory access pipeline circuit to the intermediate preblend image data stream, a blend bypass pixel for each pixel of the intermediate preblend image that does not correspond to a pixel of the non-overlapping layer image;
wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises, for each blend bypass pixel of the intermediate preblend image data stream, outputting, by the layer mixer circuit to the display data stream, a corresponding pixel of the background layer image.

### Claim 22 (depends on 21)

The method of claim 21, wherein the blend bypass pixel comprises a pixel having a pre-specified color value.

### Claim 23 (depends on 15)

The method of claim 15, wherein:
blending the intermediate preblend image data stream and the background layer image data stream as the display data stream comprises:
blending, by the layer mixer circuit, the intermediate preblend image data stream and the background layer image data stream as the display data stream within a blending stage of a plurality of blending stages, wherein each blending stage corresponds to a memory access pipeline circuit of the plurality of memory access pipeline circuits; and
outputting, by the layer mixer circuit, the display data stream of each blending stage preceding a final blending stage of the plurality of blending stages as the background layer image data stream to a successive blending stage; and
outputting the display data stream to the display device comprises outputting, by the layer mixer circuit, the display data stream of the final blending stage of the plurality of blending stages to the display device.

### Claim 24 (depends on 23)

The method of claim 23, further comprising:
receiving, by the layer mixer circuit, a memory access pipeline identifier and a blending stage indication for the first memory access pipeline circuit into a layer mixer configuration queue of a plurality of layer mixer configuration queues;
assigning, by the layer mixer circuit, a tag corresponding to the layer mixer configuration queue to the first memory access pipeline circuit based on the memory access pipeline identifier; and
responsive to receiving the intermediate preblend image data stream from the first memory access pipeline circuit, obtaining, by the layer mixer circuit, the memory access pipeline identifier and the blending stage indication from a top entry of the layer mixer configuration queue based on the tag;
wherein blending the intermediate preblend image data stream and the background layer image data stream as the display data stream within the blending stage of the plurality of blending stages is based on the blending stage indication.