# US 2014/0152690 A1 · Image processing device, image processing method, program, and integrated circuit

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0152690 A1
- Document ID: US20140152690A1
- Status: Published application
- Kind: A1
- Application number: 14/131,045
- Filed: 2012-05-08
- Published: 2014-06-05
- Earliest claimed: 2011-08-11
- Assignees: Panasonic Intellectual Property Management Co., Ltd.
- Inventors: Masato Yuda; Naoki Ootani
- Web page: https://patentgrep.com/patent/US20140152690A1

## CPC

- G06T 1/60: Memory management
- G06T 5/70: Denoising; Smoothing
- G06T 11/23: using straight lines or curves
- G06T 15/503: {Blending, e.g. for anti-aliasing}
- G06T 2200/12: involving antialiasing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An image processing device (10) includes: a representative pixel value storage unit (12) which selects one of plural sub-pixels as a representative sub-pixel, and store, into a representative pixel value storage area, a representative pixel value which is a pixel value of the representative sub-pixel; and a sub-pixel value storage unit (13) which stores, into a sub-pixel value storage area, pixel values of sub-pixels other than the representative sub-pixel, wherein the sub-pixel value storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels in the unit of processing for an edge determination unit (11).

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

An image processing device comprising:
an edge determination unit configured to determine, for each of plural pixels in a unit of processing, whether the pixel includes a predetermined edge, the plural pixels being included in image data;
a representative pixel value storage unit configured to, for each of the plural pixels, select one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and store, into a representative pixel value storage area, a representative pixel value which is a pixel value of the representative sub-pixel; and
a sub-pixel value storage unit configured to store, into a sub-pixel value storage area, pixel values of two or more of the plural sub-pixels in each of pixels determined by the edge determination unit to include the predetermined edge, the two or more of the plural sub-pixels being other than the representative sub-pixel selected by the representative pixel value storage unit,
wherein the sub-pixel value storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels in the unit of processing for the edge determination unit.

### Claim 22 (depends on 21)

The image processing device according to claim 21, further comprising:
a first memory which stores, as image data, pixel values of plural sub-pixels obtained by dividing each of pixels included in an image;
a second memory which includes the representative pixel value storage area and the sub-pixel value storage area;
an image processing unit configured to perform image processing on the image data stored in the first memory; and
a transfer unit configured to transfer at least a portion of the image data stored in the first memory between the first memory and the second memory, the transfer unit including:
the edge determination unit;
the representative pixel value storage unit; and
the sub-pixel value storage unit.

### Claim 23 (depends on 21)

The image processing device according to claim 21,
wherein the edge determination unit is configured to calculate a total number of the pixels in the unit of processing each of which includes the predetermined edge, and
the sub-pixel value storage unit is configured to increase N, with an increase in the total number of the pixels calculated by the edge determination unit, in exchange for decreasing a total number of bits in each pixel value to be stored in the representative pixel value storage area and/or the sub-pixel value storage area, N indicating a total number of pixels storable in the sub-pixel value storage area.

### Claim 24 (depends on 21)

The image processing device according to claim 21, further comprising
a pixel selection unit configured to select, when a total number of the pixels determined by the edge determination unit to include the predetermined edge exceeds N that is the natural number, N pixels from among the pixels determined to include the predetermined edge,
wherein for each of one or more pixels that are not selected as the N pixels by the pixel selection unit from among the pixels the total number of which exceeds N that is the natural number, the representative pixel value storage unit is configured to generate the representative sub-pixel by mixing pixel values of the plural sub-pixels, and store a pixel value of the generated representative sub-pixel into the representative pixel value storage area, and
for each of the N pixels selected by the pixel selection unit, the sub-pixel value storage unit is configured to store the pixel value of each sub-pixel other than the representative sub-pixel of the pixel into the sub-pixel value storage area, N being the natural number.

### Claim 25 (depends on 24)

The image processing device according to claim 24,
wherein the pixel selection unit is configured to calculate, for each of the pixels determined by the edge determination unit to include the predetermined edge, a greatest color difference between the sub-pixels included in the pixel, and select N pixels in descending order of the calculated greatest color difference.

### Claim 26 (depends on 24)

The image processing device according to claim 24,
wherein the pixel value of each of the sub-pixels includes a color value indicating a color of a pixel which includes the sub-pixel, and a depth value indicating a position of the sub-pixel in a depth direction, and
the pixel selection unit is configured to select N pixels from among the pixels determined by the edge determination unit to include the predetermined edge, in descending order of a greatest difference in the depth value between the sub-pixels.

### Claim 27 (depends on 24)

The image processing device according to claim 24,
wherein the pixel value of each of the sub-pixels includes a color value indicating a color of a pixel which includes the sub-pixel, and a depth value indicating a position of the sub-pixel in a depth direction,
the pixel selection unit is configured to, for each of the pixels determined by the edge determination unit to include the predetermined edge, calculate a greatest color difference between the sub-pixels included in the pixel, select N pixels in descending order of the calculated greatest color difference, and select N pixels from among the pixels determined by the edge determination unit to include the predetermined edge, in descending order of a greatest difference in the depth value between the sub-pixels, and
the sub-pixel value storage unit is configured to store, into different areas of the sub-pixel value storage area, a color value of each sub-pixel other than the representative sub-pixel of each of the N pixels selected by the pixel selection unit in descending order of the greatest color difference, and a depth value of each sub-pixel other than the representative sub-pixel of each of the N pixels selected by the pixel selection unit in descending order of the greatest difference in the depth value.

### Claim 28 (depends on 24)

The image processing device according to claim 24,
wherein each of pixels included in the image data is associated with a combination completed bit set to one of a first value and a second value, the first value indicating that the representative sub-pixel has been generated by mixing pixel values of plural sub-pixels included in the pixel, the second value indicating that pixel values of plural sub-pixels included in the pixel are not yet mixed, and
the pixel selection unit is configured to select N pixels from among pixels determined by the edge determination unit to include the predetermined edge and associated with the combination completed bit set to the second value.

### Claim 29 (depends on 28)

The image processing device according to claim 28, further comprising:
a first memory which stores, as image data, the pixel values of the sub-pixels obtained by dividing each of the pixels included in the image; and
an image processing unit configured to perform image processing on the image data stored in the first memory,
wherein for each of the sub-pixels, the first memory further stores a write-enable bit set to one of a third value and a fourth value, the third value indicating that the sub-pixel is used to render a polygon in immediately preceding rendering processing by the image processing unit, the fourth value indicating that the sub-pixel is not used to render a polygon in the immediately preceding rendering processing by the image processing unit, and
for each of the pixels determined by the edge determination unit to include the predetermined edge and associated with the combination completed bit set to the second value, the representative pixel value storage unit is configured to select a sub-pixel associated with the write-enable bit set to the third value as the representative sub-pixel, from among plural sub-pixels included in the pixel, or generate the representative sub-pixel by mixing pixel values of sub-pixels each associated with the write-enable bit set to the third value.

### Claim 30 (depends on 29)

The image processing device according to claim 29,
wherein the representative pixel value storage unit is configured to set, to the first value, the combination completed bit of a pixel determined by the edge determination unit to include the predetermined edge and not selected by the pixel selection unit.

### Claim 31 (depends on 21)

The image processing device according to claim 21,
wherein the edge determination unit is configured to calculate, for each of the plural pixels in the unit of processing, a greatest color difference between the sub-pixels included in the pixel, and determine a pixel the calculated greatest color difference of which exceeds a threshold value to include the predetermined edge.

### Claim 32 (depends on 21)

The image processing device according to claim 21, further comprising:
a first memory which stores, as image data, pixel values of plural sub-pixels obtained by dividing each of pixels included in an image; and
an image processing unit configured to perform image processing on the image data stored in the first memory,
wherein for each of the sub-pixels, the first memory further stores a write-enable bit set to one of a third value and a fourth value, the third value indicating that the sub-pixel is used for rendering in immediately preceding rendering processing by the image processing unit, the fourth value indicating that the sub-pixel is not used for rendering in the immediately preceding rendering processing by the image processing unit, and
the edge determination unit is configured to determine, among the plural pixels in the unit of processing, a pixel including sub-pixels associated with the write-enable bits having different values to include the predetermined edge.

### Claim 33 (depends on 24)

The image processing device according to claim 24,
wherein the pixel value of each sub-pixel includes a color value indicating a color of a pixel which includes the sub-pixel, and a depth value indicating a position of the sub-pixel in a depth direction, and
the representative pixel value storage unit is configured to select or generate the representative sub-pixel of each of the plural pixels, based on a difference in the depth value between the sub-pixels included in the pixel.

### Claim 34 (depends on 22)

The image processing device according to claim 22,
wherein the image processing unit is configured to perform anti-aliasing on the image data stored in the first memory, processing of newly creating the image data, or processing of editing an image represented by the image data.

### Claim 35 (depends on 22)

The image processing device according to claim 22,
wherein the transfer unit further includes:
a representative pixel value reading unit configured to read, from the representative pixel value storage area of the second memory, the representative pixel value of each of the plural pixels in the unit of processing;
a sub-pixel value reading unit configured to read, from the sub-pixel value storage area of the second memory, the pixel value of each sub-pixel other than the representative sub-pixel of each pixel determined to include the predetermined edge; and
a pixel load unit configured to generate a pixel value of each pixel including the predetermined edge, from the representative pixel value read by the representative pixel value reading unit and the pixel value of each sub-pixel other than the representative sub-pixel read by the sub-pixel value reading unit, generate a pixel value of each of pixels determined not to include the predetermined edge, by duplicating the representative pixel value read by the representative pixel value reading unit to each sub-pixel, and write image data including the generated pixel values of the pixels into the first memory.

### Claim 36 (depends on 35)

The image processing device according to claim 35,
wherein each of the pixels included in the image is associated with a combination completed bit set to one of a first value and a second value, the first value indicating that the representative sub-pixel has been generated by mixing pixel values of plural sub-pixels included in the pixel, the second value indicating that pixel values of plural sub-pixels included in the pixel are not yet mixed,
for each of the sub-pixels, the first memory further stores a write-enable bit set to one of a third value and a fourth value, the third value indicating that the sub-pixel is used for rendering in immediately preceding rendering processing by the image processing unit, the fourth value indicating that the sub-pixel is not used for rendering in the immediately preceding rendering processing by the image processing unit, and
the image processing unit is configured to set, to the second value, the combination completed bit of a pixel including sub-pixels each having the write-enable bit set to the third value, after rendering processing.

### Claim 37 (depends on 21)

The image processing device according to claim 21,
wherein the predetermined edge is a polygon edge.

### Claim 38

An image processing method comprising:
(a) determining, for each of plural pixels in a unit of processing, whether the pixel includes a predetermined edge, the plural pixels being included in image data;
(b) for each of the plural pixels, selecting one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and storing, into a representative pixel value storage area, a representative pixel value which is a pixel value of the representative sub-pixel; and
(c) storing, into a sub-pixel value storage area, pixel values of two or more of the plural sub-pixels in each of pixels determined in step (a) to include the predetermined edge, the two or more of the plural sub-pixels being other than the representative sub-pixel selected in step (b),
wherein the sub-pixel value storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels in the unit of processing for the edge determination unit.

### Claim 39

A non-transitory computer-readable recording medium storing a program for causing a computer to execute, for processing an image:
(a) determining, for each of plural pixels in a unit of processing, whether the pixel includes a predetermined edge, the plural pixels being included in image data;
(b) for each of the plural pixels, selecting one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and storing, into a representative pixel value storage area, a representative pixel value which is a pixel value of the representative sub-pixel; and
(c) storing, into a sub-pixel value storage area, pixel values of two or more of the plural sub-pixels in each of pixels determined in step (a) to include the predetermined edge, the two or more of the plural sub-pixels being other than the representative sub-pixel selected in step (b),
wherein the sub-pixel value storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels in the unit of processing for the edge determination unit.

### Claim 40

An integrated circuit comprising:
an edge determination unit configured to determine, for each of plural pixels in a unit of processing, whether the pixel includes a predetermined edge, the plural pixels being included in image data;
a representative pixel value storage unit configured to, for each of the plural pixels, select one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and store, into a representative pixel value storage area, a representative pixel value which is a pixel value of the representative sub-pixel; and
a sub-pixel value storage unit configured to store, into a sub-pixel value storage area, pixel values of two or more of the plural sub-pixels in each of pixels determined by the edge determination unit to include the predetermined edge, the two or more of the plural sub-pixels being other than the representative sub-pixel selected by the representative pixel value storage unit,
wherein the sub-pixel value storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels in the unit of processing for the edge determination unit.

### Claim 41

An image processing device comprising:
a representative pixel value storage unit configured to, for each of plural pixels in a unit of processing, select one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and store a pixel value of the representative sub-pixel into a first storage area; and
a sub-pixel value storage unit configured to store, into a second storage area, pixel values of two or more of the plural sub-pixels other than the representative sub-pixel, the plural pixels being included in image data,
wherein the second storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels included in the unit of processing.

### Claim 42 (depends on 41)

The image processing device according to claim 41, further comprising
an edge determination unit configured to determine, for each of the plural pixels in the unit of processing, whether the pixel includes a predetermined edge, the plural pixels being included in the image data,
wherein the sub-pixel value storage unit is configured to store, into the second storage area, pixel values of two or more of the plural sub-pixels in each of pixels determined by the edge determination unit to include the predetermined edge, the two or more of the plural sub-pixels being other than the representative sub-pixel selected by the representative pixel value storage unit.

### Claim 43 (depends on 42)

The image processing device according to claim 42,
wherein the N number of pixels are smaller in number than the plural pixels included in the unit of processing on which the edge determination unit performs the determination.

### Claim 44 (depends on 41)

The image processing device according to claim 41, further comprising
a pixel selection unit configured to select, when a total number of the pixels determined by the edge determination unit to include the predetermined edge exceeds N that is the natural number, N pixels from among the pixels determined to include the predetermined edge,
wherein for each of one or more pixels that are not selected as the N pixels by the pixel selection unit from among the pixels the total number of which exceeds N that is the natural number, the representative pixel value storage unit is configured to mix the pixel values of the plural sub-pixels to generate the representative sub-pixel,
the representative pixel value storage unit is configured to store a pixel value of the generated representative sub-pixel into the first storage area, and
the sub-pixel value storage unit is configured to store, for each of the N pixels selected by the pixel selection unit, the pixel value of each sub-pixel other than the representative sub-pixel into the second storage area.

### Claim 45 (depends on 41)

The image processing device according to claim 41,
wherein the total number of the plural pixels included in the unit of processing is a total number of the representative sub-pixels to be stored in the first storage area.

### Claim 46

An image processing method comprising:
for each of plural pixels in a unit of processing, selecting one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and storing a pixel value of the representative sub-pixel into a first storage area; and
storing, into a second storage area, pixel values of two or more of the plural sub-pixels other than the representative sub-pixel,
wherein the second storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels included in the unit of processing.

### Claim 47

A non-transitory computer-readable recording medium storing a program for causing a computer to execute, for processing an image:
for each of plural pixels in a unit of processing, selecting one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and storing a pixel value of the representative sub-pixel into a first storage area; and
storing, into a second storage area, pixel values of two or more of the plural sub-pixels other than the representative sub-pixel,
wherein the second storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels included in the unit of processing.

### Claim 48

An integrated circuit comprising:
a representative pixel value storage unit configured to, for each of plural pixels in a unit of processing, select one of plural sub-pixels obtained by dividing the pixel, as a representative sub-pixel, and store a pixel value of the representative sub-pixel into a first storage area; and
a sub-pixel value storage unit configured to store, into a second storage area, pixel values of two or more of the plural sub-pixels other than the representative sub-pixel,
wherein the second storage area stores one or more sets corresponding to N number of pixels, the one or more sets each including a pixel position and a pixel value of each sub-pixel other than the representative sub-pixel of a pixel at the pixel position, N being a natural number smaller than a total number of the plural pixels included in the unit of processing.