# US 10,504,462 B2 · Non-linear processing of two-dimensional data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,504,462 B2
- Document ID: US10504462B2
- Status: Granted
- Kind: B2
- Application number: 15/880,099
- Filed: 2018-01-25
- Published: 2019-07-25
- Granted: 2019-12-10
- Earliest claimed: 2018-01-25
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Chunhui Mei
- Web page: https://patentgrep.com/patent/US10504462B2

## CPC

- G09G 3/3607: {for displaying colours or for displaying grey scales with a specific pixel layout, e.g. using sub-pixels (display of colours in flat matrix panels other than liquid crystal displays G09G3/2003; grey scales specific for television H04N3/127)}
- G06F 12/0207: {with multidimensional access, e.g. row/column, matrix}
- G06F 12/0862: with prefetch
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 1/60: Memory management
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G06F 12/0855: Overlapped cache accessing, e.g. pipeline (G06F12/0846 takes precedence)
- G06F 2212/1016: Performance improvement
- G06F 2212/1028: Power efficiency
- G06F 2212/455: Image or video data
- G06F 2212/602: Details relating to cache prefetching
- G06F 2212/6026: Prefetching based on access pattern detection, e.g. stride based prefetch
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

A device for graphics processing includes a memory and at least one processor. The at least one processor is configured to generate image data for an image, fetch, for each two-dimensional matrix of multiple two-dimensional matrices of units of the image, a respective portion of the image data, and process each two-dimensional matrix of the multiple two-dimensional matrices based on the respective portion of the image data to generate pixel data for the image. To process each two-dimensional matrix of the multiple two-dimensional matrices, the at least one processor is configured to process multiple units arranged in a first two-dimensional matrix of the multiple two-dimensional matrices and process, after processing the multiple units arranged in the first two-dimensional matrix, multiple units arranged in a second two-dimensional matrix of the multiple two-dimensional matrices.

## Claims

### Claim 1

A method for non-linear graphics processing comprising:
generating, by a processor, image data for an image;
fetching, by the processor, for each two-dimensional matrix of a plurality of two-dimensional matrices of units of the image, a respective portion of the image data, wherein fetching, for each two-dimensional matrix of the plurality of two-dimensional matrices of units, the respective portion of the image data comprises:
fetching a first portion of the image data for processing units of a first two-dimensional matrix of the plurality of two-dimensional matrices; and
fetching, after fetching the first portion of the image data, a second portion of the image data for processing units of a second two-dimensional matrix of the plurality of two-dimensional matrices; and
processing, by the processor, each two-dimensional matrix of the plurality of two-dimensional matrices based on the respective portion of the image data to generate pixel data for the image, wherein processing each two-dimensional matrix of the plurality of two-dimensional matrices comprises:
processing a plurality of units arranged in the first two-dimensional matrix based on the first portion of the image data,
wherein the first two-dimensional matrix includes a plurality of units arranged in a first row of the image and a plurality of units arranged in a second row of the image that is adjacent to the first row of the image; and
processing, after processing the plurality of units arranged in the first two-dimensional matrix, a plurality of units arranged in the second two-dimensional matrix based on the second portion of the image data, wherein the second two-dimensional matrix includes one or more units arranged in the first row of the image and one or more units arranged in the second row of the image, wherein processing the plurality of units arranged in the second two-dimensional matrix comprises processing one or more units arranged in the second two-dimensional matrix using the first portion of image data for processing units of the first two-dimensional matrix.

### Claim 2 (depends on 1)

The method of claim 1, wherein processing each two-dimensional matrix of the plurality of two-dimensional matrices comprises:
recursively processing the image using the plurality of two-dimensional matrices.

### Claim 3 (depends on 2)

The method of claim 2, wherein recursively processing the image comprises:
processing a subset of the plurality of two-dimensional matrices,
wherein each two-dimensional matrix of the subset of the plurality of two-dimensional matrices forms a square of units of the image, and
wherein a subset of the subset of two-dimensional matrices forms a square of units of the image.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
incrementing, by the processor, an initial index value corresponding to a first unit of the image to generate a candidate index value corresponding to a candidate unit of the image;
determining, by the processor, a horizontal position of the candidate unit using odd bits of the candidate index value; and
determining, by the processor, a vertical position of the candidate unit using even bits of the candidate index value, wherein processing each two-dimensional matrix of the plurality of two-dimensional matrices is based on the horizontal position of the candidate unit and the vertical position of the candidate unit.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
determining, by the processor, that a position of the candidate unit is inside a target region of the image, wherein processing each two-dimensional matrix of the plurality of two-dimensional matrices comprises processing the candidate unit in response to determining that the position of the candidate unit is inside the target region.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining that the position of the candidate unit is inside the target region of the image comprises:
comparing, by the processor, the horizontal position of the candidate unit to a horizontal threshold value for the target region; and
comparing, by the processor, the vertical position of the candidate unit to a vertical threshold value for the target region.

### Claim 7 (depends on 4)

The method of claim 4, further comprising:
in response to determining that a position of the candidate unit is outside a target region of the image, determining, by the processor, a subsequent index value corresponding to a subsequent candidate unit for the image that is within the target region, wherein processing each two-dimensional matrix of the plurality of two-dimensional matrices comprises:
in response to determining that the position of the candidate unit is outside the target region:
refraining from processing the candidate unit; and
processing the subsequent candidate unit.

### Claim 8 (depends on 7)

The method of claim 7, wherein determining the subsequent index value comprises:
selecting a least-significant odd bit of the candidate index value that has a value of ‘1’ as a selected odd bit of the candidate index value;
selecting a least-significant even bit of the candidate index value that has a value of ‘0’ and that is positioned in a more-significant bit than the selected odd bit as a selected even bit of the candidate index value;
setting the selected even bit to ‘1’; and
resetting all bits that are less-significant than the selected even bit to ‘0’.

### Claim 9 (depends on 1)

The method of claim 1, wherein the processor is a graphics processing unit and wherein fetching, for each two-dimensional matrix of the plurality of two-dimensional matrices of units, the respective portion of the image data comprises fetching, from system memory, for storage at graphics memory for the graphics processing unit, the respective portion of the image data.

### Claim 10 (depends on 1)

The method of claim 1, wherein fetching, for each two-dimensional matrix of the plurality of two-dimensional matrices of units, the respective portion of the image data comprises fetching image data for a neighboring unit to process a particular unit of a respective two-dimensional matrix of units, the neighboring unit being arranged above or below the particular unit and outside of the respective two-dimensional matrix.

### Claim 11 (depends on 1)

The method of claim 1, wherein processing each two-dimensional matrix of the plurality of two-dimensional matrices comprises one or more of motion blur processing, depth of field processing, or Gaussian blur processing.

### Claim 12 (depends on 1)

The method of claim 1, wherein fetching, for each two-dimensional matrix of the plurality of two-dimensional matrices of units, the respective portion of the image data comprises:
determining, by the processor, for each unit of each two-dimensional matrix of the plurality of two-dimensional matrices, two-dimensional image data for processing a respective unit, wherein processing each two-dimensional matrix of the plurality of two-dimensional matrices comprises:
processing each unit of the image using the respective two-dimensional image data.

### Claim 13 (depends on 12)

The method of claim 12, wherein the two-dimensional image data comprises one or more of: red, green, blue (RGB) data; red, green, blue, alpha (RGBA) data; chroma data; luma data; or alpha data.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
determining, by the processor, for each two-dimensional matrix of the plurality of two-dimensional matrices of units, a size of the respective portion of the image data based on one or more of a cache size of the processor or an image format for the image.

### Claim 15

A device for non-linear graphics processing comprising:
a memory; and
at least one processor configured to:
generate image data for an image;
fetch, for each two-dimensional matrix of a plurality of two-dimensional matrices of units of the image, for storage at the memory, a respective portion of the image data, wherein, to fetch, for each two-dimensional matrix of the plurality of two-dimensional matrices of units, the respective portion of the image data, the at least one processor is configured to:
fetch a first portion of the image data for processing units of a first two-dimensional matrix of the plurality of two-dimensional matrices; and
fetch, after fetching the first portion of image data, a second portion of the image data for processing units of a second two-dimensional matrix of the plurality of two-dimensional matrices; and
process each two-dimensional matrix of the plurality of two-dimensional matrices based on the respective portion of the image data to generate pixel data for the image, wherein, to process each two-dimensional matrix of the plurality of two-dimensional matrices, the at least one processor is configured to:
process a plurality of units arranged in the first two-dimensional matrix based on the first portion of the image data, wherein the first two-dimensional matrix includes a plurality of units arranged in a first row of the image and a plurality of units arranged in a second row of the image that is adjacent to the first row of the image; and
process, after processing the plurality of units arranged in the first two-dimensional matrix, a plurality of units arranged in the second two-dimensional matrix based on the second portion of the image data, wherein the second two-dimensional matrix includes one or more units arranged in the first row of the image and one or more units arranged in the second row of the image, wherein, to process the plurality of units arranged in the second two-dimensional matrix, the at least one processor is configured to process one or more units arranged in the second two-dimensional matrix using the first portion of image data for processing units of the first two-dimensional matrix.

### Claim 16 (depends on 15)

The device of claim 15, wherein, to process each two-dimensional matrix of the plurality of two-dimensional matrices, the at least one processor is configured to:
recursively process the image using the plurality of two-dimensional matrices.

### Claim 17 (depends on 16)

The device of claim 16, wherein, to recursively process the image, the at least one processor is configured to:
process a subset of the plurality of two-dimensional matrices,
wherein each two-dimensional matrix of the subset of the plurality of two-dimensional matrices forms a square of units of the image, and
wherein a subset of the subset of two-dimensional matrices forms a square of units of the image.

### Claim 18 (depends on 15)

The device of claim 15, wherein the at least one processor is configured to:
increment an initial index value corresponding to a first unit of the image to generate a candidate index value corresponding to a candidate unit of the image;
determine a horizontal position of the candidate unit using odd bits of the candidate index value; and
determine a vertical position of the candidate unit using even bits of the candidate index value, wherein, to process each two-dimensional matrix of the plurality of two-dimensional matrices, the at least one processor is configured to process each two-dimensional matrix of the plurality of two-dimensional matrices based on the horizontal position of the candidate unit and the vertical position of the candidate unit.

### Claim 19 (depends on 18)

The device of claim 18, wherein the at least one processor is configured to:
determine that a position of the candidate unit is inside a target region of the image, wherein, to process each two-dimensional matrix of the plurality of two-dimensional matrices, the at least one processor is configured to process the candidate unit in response to determining that the position of the candidate unit is inside the target region.

### Claim 20 (depends on 19)

The device of claim 19, wherein, to determine that the position of the candidate unit is inside the target region of the image, the at least one processor is configured to:
compare the horizontal position of the candidate unit to a horizontal threshold value for the target region; and
compare the vertical position of the candidate unit to a vertical threshold value for the target region.

### Claim 21 (depends on 18)

The device of claim 18, wherein the at least one processor is configured to:
in response to determining that a position of the candidate unit is outside a target region of the image, determine a subsequent index value corresponding to a subsequent unit for the image that is within the target region, wherein, to process each two-dimensional matrix of the plurality of two-dimensional matrices, the at least one processor is configured to:
in response to determining that the position of the candidate unit is outside the target region:
refrain from processing the candidate unit; and
process the subsequent candidate unit.

### Claim 22 (depends on 21)

The device of claim 21, wherein, to determine the subsequent index value, the at least one processor is configured to:
select a least-significant odd bit of the candidate index value that has a value of ‘1’ as a selected odd bit of the candidate index value;
select a least-significant even bit of the candidate index value that has a value of ‘0’ and that is positioned in a more-significant bit than the selected odd bit as a selected even bit of the candidate index value;
set the selected even bit to ‘1’; and
reset all bits that are less-significant than the selected even bit to ‘0’.

### Claim 23 (depends on 15)

The device of claim 15, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

### Claim 24 (depends on 15)

The device of claim 15, wherein the device comprises at least one of:
an integrated circuit;
a microprocessor; or
a wireless communication device.

### Claim 25

A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to:
generate image data for an image;
fetch, for each two-dimensional matrix of a plurality of two-dimensional matrices of units of the image, a respective portion of the image data, wherein, to fetch, for each two-dimensional matrix of the plurality of two-dimensional matrices of units, the respective portion of the image data, the instructions further cause the at least one processor to:
fetch a first portion of the image data for processing units of a first two-dimensional matrix of the plurality of two-dimensional matrices; and
fetch, after fetching the first portion of image data, a second portion of the image data for processing units of a second two-dimensional matrix of the plurality of two-dimensional matrices; and
process each two-dimensional matrix of the plurality of two-dimensional matrices based on the respective portion of the image data to generate pixel data for the image, wherein, to process each two-dimensional matrix of the plurality of two-dimensional matrices, the instructions further cause the at least one processor to:
process a plurality of units arranged in the first two-dimensional matrix based on the first portion of the image data, wherein the first two-dimensional matrix includes a plurality of units arranged in a first row of the image and a plurality of units arranged in a second row of the image that is adjacent to the first row of the image; and
process, after processing the plurality of units arranged in the first two-dimensional matrix, a plurality of units arranged in the second two-dimensional matrix based on the second portion of the image data, wherein the second two-dimensional matrix includes one or more units arranged in the first row of the image and one or more units arranged in the second row of the image, wherein, to process the plurality of units arranged in the second two-dimensional matrix, the instructions further cause the at least one processor to process one or more units arranged in the second two-dimensional matrix using the first portion of image data for processing units of the first two-dimensional matrix.