# US 2008/0186325 A1 · Pre-Subpixel Rendered Image Processing In Display Systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0186325 A1
- Document ID: US20080186325A1
- Status: Published application
- Kind: A1
- Application number: 11/910,649
- Filed: 2006-04-04
- Published: 2008-08-07
- Earliest claimed: 2005-04-04
- Assignees: Clairvoyante, Inc
- Inventors: Michael Francis Higgins; Seok Jin Han; Candice Hellen Brown Elliott
- Web page: https://patentgrep.com/patent/US20080186325A1

## CPC

- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G09G 3/2003: {Display of colours (specific for liquid crystal displays G09G3/3607)}
- G09G 5/024: {using colour registers, e.g. to control background, foreground, surface filling (G09G5/06 takes precedence)}
- G09G 5/026: {Control of mixing and/or overlay of colours in general (G09G5/022 and G09G5/024 take precedence)}
- G09G 5/28: for enhancement of character form, e.g. smoothing
- H04N 9/76: for mixing of colour signals (H04N9/75 takes precedence)
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G09G 5/14: Display of multiple viewports
- G09G 5/24: Generation of individual character patterns
- G09G 2300/0443: with several sub-pixels for the same colour in a pixel, not specifically used to display gradations (G09G3/364 takes precedence)
- G09G 2300/0452: Details of colour pixel setup, e.g. pixel composed of a red, a blue and two green components
- G09G 2320/0276: for the purpose of adaptation to the characteristics of a display device, i.e. gamma correction
- G09G 2340/0457: Improvement of perceived resolution by subpixel rendering
- G09G 2340/06: Colour space transformation
- G09G 2340/10: Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
- G09G 2360/18: Use of a frame buffer in a display terminal, inclusive of the display panel
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Display systems and image processing methods process pre-subpixel rendered images embedded in input color image data. The display systems include a pre-subpixel rendered (P-SPR) image detector that detects locations of a marking code that marks the portion of the input data that has been pre-subpixel rendered and which is ready for direct display. Several display system embodiments comprise first and second image data paths; the input data that requires subpixel rendering proceeds along the first path while the P-SPR image data proceeds along the second path. Another display system embodiment processes the combined input and P-SPR data along a single data path. Techniques for marking and detecting P-SPR image data using two distinct marking codes are presented in the context of the subpixel layout of the display. Techniques for using P-SPR data to display high-quality graphical symbols (e.g., font glyphs) are suitable for small, low-cost display devices.

## Claims

### Claim 1

A display device having a display panel configured with a subpixel layout; the display device comprising:
input circuitry for receiving input image data including an image data subset indicating a plurality of pre-subpixel rendered image data values in conformance with, and ready for display on, the subpixel layout of the display panel;
a pre-subpixel rendering detection component for detecting an input image location of each of the plurality of pre-subpixel rendered image data values;
a subpixel rendering component for performing a subpixel rendering operation on the input image data to produce subpixel rendered image data values in conformance with, and ready for display on, the subpixel layout of the display panel; and
an output data selection component for selecting an output color data value for each output location on the display panel; the output data selection component selecting one of the subpixel rendered image data value and the pre-subpixel rendered image data value according to the input image locations detected by the pre-subpixel rendering detection component.

### Claim 2 (depends on 1)

The display device of claim 1 wherein the subpixel rendering component includes a color selection component for removing a color value from input to the subpixel rendering operation when the color value is detected to be in an input image location indicating a pre-subpixel rendered image data value by the pre-subpixel rendering detection component.

### Claim 3 (depends on 2)

The display device of claim 2 wherein the output data selection component for selecting an output color data value for each output location on the display panel is included in the subpixel rendering component

### Claim 4 (depends on 1)

The display device of claim 1 wherein the input image data proceeds along a first image data path to the pre-subpixel rendering detection component and along a second image data path to the subpixel rendering component; and
wherein the output data selection component receives the input image locations detected by the pre-subpixel rendering detection component from the first image data path and receives the subpixel rendered image data values from the second image data path.

### Claim 5 (depends on 4)

The display device of claim 4 wherein the first image data path includes a line delay buffer for storing the input image locations detected by the pre-subpixel rendering detection component.

### Claim 6 (depends on 4)

The display device of claim 4 further including a shared line buffer for storing the input image data proceeding along the second image data path to the subpixel rendering component and the input image locations detected by the pre-subpixel rendering detection component along the first data path.

### Claim 7 (depends on 1)

The display device of claim 1 further including
a gamut mapping component for performing a gamut mapping operation on the input image data to produce gamut-mapped image data values for input to the subpixel rendering component; and
a gamut data selection component for selecting a gamut-mapped color data value for each input image location; the gamut data selection component selecting, for input to the subpixel rendering component, one of the gamut-mapped image data value and the pre-subpixel rendered image data value according to the input image locations detected by the pre-subpixel rendering detection component.

### Claim 8 (depends on 7)

The display device of claim 7 wherein each gamut-mapped image data value produced by the gamut mapping operation includes a luminosity value;
wherein the display device further includes a p-spr indicator storing component for storing a p-spr indicator value in a portion of the luminosity value for the gamut-mapped image data value for an input image location when the pre-subpixel rendering detection component determines that a pre-subpixel rendered image data value is present in the input image location; and
wherein the output data selection component selects one of the subpixel rendered image data value and the pre-subpixel rendered image data value according to the p-spr indicator value.

### Claim 9 (depends on 1)

The display device of claim 1 wherein the pre-subpixel rendering detection component detects a marking code embedded in the input image data to determine whether a pre-subpixel rendered image data value is present in an input image location.

### Claim 10 (depends on 9)

The display device of claim 9 wherein the marking code is embedded in the image data subset and marks the extent of the input image locations indicating the pre-subpixel rendered image data values.

### Claim 11 (depends on 9)

The display device of claim 9 wherein the input image data includes color data values indicating at least first, second and third primary colors; and wherein the marking code is a color key code comprising at least one color value in the first primary color and at least one color value in the second primary color.

### Claim 12 (depends on 10)

The display device of claim 10 wherein the marking code is a color key code comprising at least two color values in the first primary color.

### Claim 13

A display device having a display panel configured with a subpixel layout; the display device comprising:
input circuitry for receiving input image data including an image data subset indicating a plurality of pre-subpixel rendered image data values in conformance with, and ready for display on, the subpixel layout of the display panel;
a pre-subpixel rendering detection component for detecting an input image location of each of the plurality of pre-subpixel rendered image data values; the pre-subpixel rendering detection component setting a p-spr data variable for each input image location indicating whether the input image location is a pre-subpixel rendered image data value;
a gamut mapping component for performing a gamut mapping operation on the input image data to produce gamut-mapped image data values for input to the subpixel rendering component;
a subpixel rendering component for performing a subpixel rendering operation on the gamut-mapped image data values to produce subpixel rendered image data values in conformance with, and ready for display on, the subpixel layout of the display panel;
a memory component for buffering the gamut-mapped image data values for input to the subpixel rendering component;
an output data selection component for selecting an output color data value for each output location on the display panel; the output data selection component selecting one of the subpixel rendered image data value and the pre-subpixel rendered image data value according to the value of the p-spr data variable for each input image location; and
a line delay buffer for storing the p-spr data variables for each input image location prior to input to the output data selection component.

### Claim 14 (depends on 13)

The display device of claim 13 further including a gamut data selection component for selecting a gamut-mapped color data value for each input image location; the gamut data selection component selecting, for input to the subpixel rendering component, one of the gamut-mapped image data value and the pre-subpixel rendered image data value according to the p-spr data variables for each input image location.

### Claim 15 (depends on 13)

The display device of claim 13 wherein the memory component and the line delay buffer share a common memory component.

### Claim 16 (depends on 13)

The display device of claim 13 wherein the input image data further comprises a second image data subset comprising a marking code embedded in the plurality of pre-subpixel rendered image data values for identifying the pre-subpixel rendered image data values;
wherein selected ones of the of pre-subpixel rendered image data values have been swapped from original positions with data values included in the marking code to swapped positions in the input image data; and
wherein the display device further includes a data swapping component for swapping the selected ones of the of pre-subpixel rendered image data values with data values included in the marking code from swapped positions in the input image data to the original positions prior to input to the output data selection component for selecting an output color data value.

### Claim 17 (depends on 13)

The display device of claim 13 further including first and second data swapping components; the first data swapping component swapping selected ones of the pre-subpixel rendered image data values from original positions in the image data subset with data values included in the marking code to swapped positions in the image data subset; the second data swapping component swapping the selected ones of the of pre-subpixel rendered image data values with data values included in the marking code from swapped positions in the input image data to the original positions prior to input to the output data selection component for selecting an output color data value.

### Claim 18

A display device having a display panel configured with a subpixel layout; the display device comprising:
input circuitry for receiving input image data including an image data subset indicating a plurality of pre-subpixel rendered image data values in conformance with, and ready for display on, the subpixel layout of the display panel;
a pre-subpixel rendering detection component for detecting an input image location of each of the plurality of pre-subpixel rendered image data values; the pre-subpixel rendering detection component setting a p-spr data variable for each input image location indicating whether the input image location is a pre-subpixel rendered image data value; and
a subpixel rendering component for performing a subpixel rendering operation on the input image data to produce subpixel rendered image data values in conformance with, and ready for display on, the subpixel layout of the display panel; the subpixel rendering component including
a color selection component for removing a color value from input to the subpixel rendering operation when the p-spr data variable for the input image location of the color value indicates that the color value is a pre-subpixel rendered image data value; and
an output data selection component for selecting an output color data value for each output location on the display panel; the output data selection component selecting one of the subpixel rendered image data value and the pre-subpixel rendered image data value according to the p-spr data variable for the input image location.

### Claim 19

A display system comprising an input gamma component, a subpixel rendering processor, and an output gamma component, the display system further comprising:
a pre-subpixel rendering detection component that accepts input image data and detects a pre-subpixel rendering code indicating a section of the input image data which does not need subpixel rendering;
wherein the input image data that does not need subpixel rendered is processed separately from the input gamma processor.

### Claim 20 (depends on 19)

The display system of claim 19 further comprising a multiplexor that selects among a set of processed image data, the set of processed image data including pre-subpixel rendered image data and image data which is subpixel rendered by the display system.

### Claim 21 (depends on 19)

The display system of claim 19 farther comprising a color swapping unit that swaps color image data in pre-subpixel rendered image data.

### Claim 22 (depends on 19)

The display system of claim 19 wherein the pre-subpixel image data comprises a code that differentiates between input image data which is pre-subpixel rendered and input image data that is not pre-subpixel rendered.

### Claim 23

A display system comprising:
a pre-subpixel rendering detection component that accepts input image data and detects a pre-subpixel rendering code indicating a portion of the input image data indicating pre-subpixel rendered image data which does not need subpixel rendering; the input image data proceeding along a first image data path;
a subpixel rendering processor for subpixel rendering the input image data along a second image data path.
a color swapping component for swapping color image data included in the pre-subpixel rendered image data proceeding along the first image data path;
wherein processed image data in the first and the second image data paths are multiplexed for output depending upon whether the presubpixel rendering code is present or absent.

### Claim 24 (depends on 23)

The display system of claim 23 further comprising a line buffer capable of storing image data from the first and the second image data paths.

### Claim 25 (depends on 24)

The display system of claim 24 wherein the line buffer further stores a code output by the pre-subpixel rendering detection component differentiating between input image data which is pre-subpixel rendered image data from input image data that is not pre-subpixel rendered image data.

### Claim 26 (depends on 23)

The display system of claim 23 wherein the pre-subpixel rendering detection component detects the pre-subpixel rendering code in the input image data in input row order, one row at a time.

### Claim 27

An image processing method for rendering characters in a font onto a display device having a subpixel layout and capable of detecting pre-subpixel rendered image data; the method comprising:
performing a subpixel rendering operation on image data values representing each character in the font to produce pre-subpixel rendered glyph image data indicating a glyph ready for display on the display device;
marking the pre-subpixel rendered glyph image data with a marking code to produce marked pre-subpixel rendered glyph image data; the marking code distinguishing the pre-subpixel rendered glyph image data from image data that has not been subpixel rendered;
storing the marked pre-subpixel rendered graphical symbol image data in a memory device; and
in response to a signal requesting display of a character in the font, outputting the marked pre-subpixel rendered glyph image data indicating the character to the display device.

### Claim 28 (depends on 27)

The image processing method of claim 27 further comprising storing marked pre-subpixel rendered glyph image data indicating a version of each glyph for each logical pixel in the subpixel layout of the display at which a rendered glyph may be positioned.

### Claim 29 (depends on 27)

The image processing method of claim 27 further comprising storing marked pre-subpixel rendered glyph image data indicating a version of each glyph for each subpixel in the subpixel layout of the display at which a rendered glyph may be positioned.

### Claim 30 (depends on 27)

The image processing method of claim 27 further including, when outputting the pre-subpixel rendered glyph image data indicating the character to the display device, adding marked image data to the marked pre-subpixel rendered glyph image data in order to position a rendered glyph at one of a subpixel or logical pixel starting position.

### Claim 31 (depends on 27)

The method of claim 27 wherein the font is an outline font and each character in the font is defined as a set of lines and curves.

### Claim 32

An image processing method for displaying a pre-subpixel rendered glyph stored in a memory device as a density map of pixels each indicating density values specifying a mixing percentage of each of a foreground color and a background color; the method comprising:
receiving an input signal to display the pre-subpixel rendered glyph;
retrieving the pre-subpixel rendered glyph from the memory device;
for each pixel of the pre-subpixel rendered glyph, combining the mixing percentages of an input foreground color value and an input background color value according to the density values stored in the density map to produce color image data values indicating a blended color glyph; and
outputting the color image data values indicating the blended color glyph directly to a display device without further subpixel rendering.

### Claim 33 (depends on 32)

The image processing method of claim 32 wherein the input foreground color value and the input background color value are specified in a linear color space.

### Claim 34 (depends on 32)

The image processing method of claim 32 further including performing a gamut mapping operation on the input foreground color value and the input background color value to produce image data values specified in a color space of the display device.

### Claim 35 (depends on 32)

The image processing method of claim 32 further including, when one of the input foreground color value and the input background color value is specified between a full-off black color and a full-on bright color, performing an input gamma operation to at least one of the input foreground color value and the input background color value to produce linear color space values prior to combining the mixing percentages of the input foreground color value and the input background color value.

### Claim 36 (depends on 32)

The image processing method of claim 32 wherein the input signal to display the pre-subpixel rendered glyph signal includes the foreground color value and the background color value.

### Claim 37 (depends on 32)

The image processing method of claim 32 wherein the input signal to display the pre-subpixel rendered glyph signal includes the input foreground color value; and wherein the image processing method further includes determining the input background color value from a destination image region in a target output image to be displayed on the display device.

### Claim 38 (depends on 37)

The image processing method of claim 37 wherein the blended color glyph is displayed in the destination image region of the target output image such that the blended color glyph overlays the destination image region.

### Claim 39 (depends on 37)

The image processing method of claim 37 wherein determining the input background color value from the destination image region in the target output image includes:
retrieving an input source image data subset from an input source image from which the target output image was produced; the input source image data subset corresponding to the destination image region in the target output image;
performing a gamut mapping operation on the input source image data subset to produce gamut-mapped source image data indicating color values specified in a color space of the display device; and
performing a sub-pixel rendering operation gamut-mapped source image data to produce the input background color value.

### Claim 40 (depends on 37)

The image processing method of claim 37 wherein determining the input background color value from the destination image region in the target output image comprises retrieving the input background color value from a frame buffer in the display device.

### Claim 41 (depends on 32)

The image processing method of claim 32 correcting the color image data values indicating the blended color glyph for output gamma prior to outputting the blended color glyph.

### Claim 42 (depends on 32)

The image processing method of claim 32 wherein the display device is capable of detecting pre-subpixel rendered image data, and wherein the method further includes, prior to outputting the blended glyph directly to the display,
marking the blended glyph with a marking code; and
embedding the marked blended color glyph in an image data stream; the marking code identifying the blended color glyph to the display as pre-subpixel rendered image data ready for display without further subpixel rendering.
